Automatic image-based recommendations using a color palette

ABSTRACT

Systems and methods are described that recommend images, items, and/or metadata based at least in part on a reference color palette or reference color name. A color name can be converted into a representation of the color name in a color space. The reference color can be used to identify images that contain the reference color. The identified images and associated metadata can be analyzed, sorted and provided as an ordered list of items. Systems and methods are also described that identify items that contain colors affiliated with the reference color. Systems and methods are also described that validate color identifier information in metadata associated with an image. Systems and methods are also described that identify non-color specific keywords associated with the reference color.

INCORPORATION BY REFERENCE TO ANY PRIORITY APPLICATIONS

Any and all applications for which a foreign or domestic priority claimis identified in the Application Data Sheet as filed with the presentapplication are hereby incorporated by reference under 37 CFR 1.57.

This application is a continuation of U.S. patent application Ser. No.14/316,598, entitled “AUTOMATIC IMAGE-BASED RECOMMENDATIONS USING ACOLOR PALETTE” and filed on Jun. 26, 2014, the disclosure of which isincorporated herein by reference.

BACKGROUND

Items that create a visual impression generally utilize color, at leastin part, to generate the visual impression. A palette of colors can beused, e.g., with clothes, artwork, images, video, and other visual mediaor items to provide a certain or desired look and feel. A person maywant to include a particular color or adhere to a color scheme whendecorating a room, buying clothes, accessorizing a vehicle, or the like.The person may utilize a network resource, such as a commerce networksite, to browse items associated with different colors. The items may bedepicted in images, (e.g., photographs and videos) presented via thenetwork site.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects and advantages of the embodiments provided herein are describedwith reference to the following detailed description in conjunction withthe accompanying drawings. Throughout the drawings, reference numbersmay be re-used to indicate correspondence between referenced elements.The drawings are provided to illustrate example embodiments describedherein and are not intended to limit the scope of the disclosure.

FIG. 1 is a block diagram illustrating an embodiment of an operatingenvironment that includes an image processing service for analyzingcolor images based at least in part on a color palette, a recommendationservice, and an affiliated color service.

FIG. 2 is a block diagram illustrating an embodiment of examplecomponents of a computing system utilized in accordance with theoperating environment of FIG. 1.

FIG. 3 is a flow diagram illustrating an embodiment of a routineimplemented by the recommendation service for providing recommendationsbased at least in part on a color palette.

FIG. 4 is a flow diagram illustrating an embodiment of a routineimplemented by the image processing service for validating color namesin image metadata based at least in part on input color names.

FIG. 5 is a flow diagram illustrating an embodiment of a routineimplemented by the recommendation service for determining keywords basedat least in part on a color palette determined from input color names.

FIG. 6 is a flow diagram illustrating an embodiment of a routineimplemented by the recommendation service for generating an affiliatedcolor palette based at least in part on an input color name.

FIG. 7 is a flow diagram illustrating an embodiment of a routineimplemented by the recommendation service for generating an affiliatedcolor palette based at least in part on colors determined using an inputcolor name and a color threshold value.

FIG. 8 is a flow diagram illustrating an embodiment of a routineimplemented by an affiliated color service for generating an orderedlist of affiliated colors, the routine implemented by an affiliatedcolor service.

FIG. 9 illustrates an example ordered list of affiliated colorsgenerated by the example routine of FIG. 8.

FIG. 10 is a flow diagram illustrating an embodiment of an affiliatedcolor palette generation routine implemented by the affiliated colorservice.

FIG. 11 illustrates example affiliated color palettes generated by theroutine of FIG. 10.

DETAILED DESCRIPTION

Generally described, aspects of the present disclosure relate toproviding search and recommendation tools that use color names and/orcolor palettes to identify, within a data store, relevant images, colorpalettes, keywords, metadata and the like. The present disclosureincludes systems and methods configured to process a color or colorpalette name, identify a color or color palette from that name, andsearch among a data store of metadata, images, colors, keywords, and thelike for relevant information.

Often a user visiting a networked site, such as a Web site, may wish tobrowse a category of items (e.g., goods and/or services) that include aparticular color or colors. In certain instances, users visiting thesite may wish to locate an accessory or other item having a color thatcoordinates with a color palette, such as an item of clothing,furniture, appliances, housewares, etc. The color palette may be builtfrom an initial color or colors by adding affiliated colors to theinitial color(s). In other instances, a user may want to know whatnon-color specific keywords are associated with a color palette. Incertain instances, a merchandiser may want to use a color palette toassemble a set of coordinating items from a catalog of items. Inaddition, it may be useful to validate image metadata, such as colornames, to ensure that the color names in the metadata accuratelycorrespond to known, expected or standard color names.

One or more of the various use cases discussed above are addressed byone or more embodiments disclosed herein. Aspects of the presentdisclosure relate to generating color-related recommendations andutilizing color names and/or color palettes that are collections ofrepresentative colors, each optionally associated with weight or othermetadata, in generating recommendations. Yet other aspects of thepresent disclosure validate metadata associated with an image and/orgenerate metadata, such as color names, for an image. Yet furtheraspects of the present disclosure relate to identifying non-colorspecific keywords from a color name. Additional aspects of the presentdisclosure relate to generating affiliated color palettes from a colorname.

In accordance with an illustrative embodiment, a recommendation serviceobtains a color palette and identifies color images depicting one ormore items, a design, a scene, or the like, which corresponds to thecolor palette. The color palette can be input by a user, a host of acommerce or other network site, a merchandise provider, vendor, or otherparties; generated from an image; generated from a color name; or thelike. The color images may include metadata that provides additionalinformation about the image, such as an item included in the image. Themetadata can be used to provide search results and recommendations basedon the color palette.

In accordance with another illustrative embodiment, a recommendationservice can receive a reference color name and determine one or moreimages that contain the reference color name. The recommendation servicecan obtain metadata associated with the identified images and extractone or more color identifiers from the metadata. The recommendationservice can then compare a reference color corresponding to thereference color name to colors corresponding to the one or more coloridentifiers in the metadata to validate the color identifiers. Wherethere are differences or where there is a lack of color identifiers, therecommendation service can modify the metadata to correct the coloridentifier information.

In accordance with another illustrative embodiment, a recommendationservice can receive a reference color name and determine one or morekeywords associated with the reference color name. The recommendationservice can obtain a color name, such as through a text query, determinea color or colors in a color space that correspond to the color name,and determine keywords that are associated with the colors. The list ofkeywords can be sorted based on keyword rankings or weights and providedto a user, system or service.

In accordance with another illustrative embodiment, a color name can beused to identify one or more colors affiliated with the colorcorresponding to the color name. For example, a reference color name canbe received and a plurality of palettes can be identified from a datastore of palettes that contain that color (or a sufficiently similarcolor). From those palettes, a list of affiliated colors can begenerated by identifying the other colors in the palettes. For eachaffiliated color in the list, a weight can be assigned based on theranking, rating, and/or number of votes the containing palette hasreceived. The list of affiliated colors can be ordered based on theassigned weights. The program or user can select an affiliated colorfrom the ordered or weighted list to add to a custom color palettecontaining the reference color. When the selected affiliated color isadded to the palette (or otherwise associated with the original color),a new list of affiliated colors can be generated based at least in parton the colors in the palette that allows the program or user to continueto build the color palette (or the associated grouping of affiliatedcolors).

In a further illustrative embodiment, the reference color can have acolor distance threshold such that affiliated colors are identifiedwhere an affiliated color palette contains at least one color within thecolor distance threshold. Once the affiliated colors and/or affiliatedcolor palette is determined, images containing the affiliated colors canbe identified by comparing the colors in the affiliated color palettewith colors extracted from an image. Metadata associated with theidentified images can be provided where the metadata may provideadditional information about the items in the images.

While a retail or commerce environment is often used as an examplebelow, it will be appreciated that image, data, and/or coloridentification from keyword and/or text searching of color palettes, asdisclosed herein, may be used in a variety of environments other than aretail environment. For example, aspects of the present disclosure, insome embodiments, may be used and/or implemented to efficiently identifyor surface images and/or colors related to color palettes within anyuser interface, page, video, electronic book and/or other electroniccontent. In addition, aspects of the present disclosure, in someembodiments, may be used by consumers, merchandisers, designers,architects, artists, landscapers, developers, gamers, students, etc. forvirtually any purpose. Without limitation, aspects of the presentdisclosure may be used for identifying images and/or colors related tocolor palettes in social networking contexts, digital photo albums,digital news articles, and/or visual bookmarking contexts. Forillustrative purposes, item images are often described below in thecontext of items listed in an electronic catalog. Alternatively, inother embodiments, item images that may be presented according to thesystems and methods described herein may include advertisements, imagesin news articles, editorial content, videos, classified listings,auction listings and/or any other content that may be electronicallypresented to a user.

Overview of an Example Embodiment of Color Palette Services

FIG. 1 illustrates an embodiment of an operating environment 100 thatcan implement the features described herein in the context of an examplerecommendation service 102. In some embodiments, the operatingenvironment 100 includes the recommendation service 102, an imageprocessing service 104, an affiliated color service 105, a commerceservice 106, a palette data store 110, a dictionary data store 112, anetwork 120, an item data store 130, third party users 140, and colorpalette providers 150. In some embodiments, various components of theoperating environment 100 are communicatively interconnected with oneanother via the network 120. The operating environment 100 may includedifferent components, a greater or fewer number of components, and canbe structured differently. For example, there can be more than one datastore or other computing devices in connection with the recommendationservice 102. As another example, components of the operating environment100 may communicate with one another with or without the network 120.

The image processing service 104 can correspond to any system capable ofperforming the associated processes described herein. The imageprocessing service 104 may be implemented by one or more computingdevices. For example, the image processing service 104 may beimplemented by computing devices that include one or more processors toexecute one or more instructions, memory, and communication devices totransmit and receive data over the network 120. In some embodiments, theimage processing service 104 is implemented on one or more backendservers capable of communicating over a network. In other embodiments,the image processing service 104 is implemented by one or more virtualmachines in a hosted computing environment (e.g., a “cloud” computingenvironment). The hosted computing environment may include one or moreprovisioned and released computing resources, which computing resourcesmay include computing, networking or storage devices.

In one aspect, the image processing service 104 can correspond to one ormore applications that perform, individually or in combination, theimage processing functions described herein, including imagepre-processing, color matching, color thresholding, color palettedetermination, etc. In another aspect, the image processing service 104may be configured to store or update palettes at the palette data store110. In some embodiments, the image processing service 104 is associatedwith a network or network-based merchandise provider, vendor, or otherparties. The image processing service 104 may access and process imagesfrom the item data store 130, provided by merchandisers for posting on acommerce network site on a network page that consumers can use topurchase the item, provided by consumers, provided by third party imagedata stores, or provided by other image sources. The image processingservice 104 is communicatively connected to the palette data store 110.

The affiliated color service 105 can correspond to any system capable ofperforming the associated processes described herein. The affiliatedcolor service 105 may be implemented by one or more computing devices.For example, the affiliated color service 105 may be implemented bycomputing devices that include one or more processors to execute one ormore instructions, memory, and communication devices to transmit andreceive data over the network 120. In some embodiments, the affiliatedcolor service 105 is implemented on one or more backend servers capableof communicating over a network. In certain embodiments, the affiliatedcolor service 105 is implemented by one or more virtual machines in ahosted computing environment.

In some aspects, the affiliated color service 105 can correspond to oneor more applications that perform, individually or in combination, thefunctions described herein, including determining affiliated colors,assigning weights to affiliated colors, normalizing weights ofaffiliated colors, clustering or combining colors based on colordistances, determining color similarity thresholds, updating affiliatedcolors based at least in part on updated palettes, etc. In certainaspects, the affiliated color service 105 may be configured to store orupdate palettes at the palette data store 110, and thus may becommunicatively connected to the palette data store 110. In someembodiments, the affiliated color service 105 is associated with anetwork or network-based merchandise provider, vendor and/or otherparties.

The recommendation service 102 can correspond to any system capable ofperforming the associated processes described herein. The recommendationservice 102 may be implemented by one or more computing devices. Forexample, the recommendation service 102 may be implemented by computingdevices that include one or more processors to execute one or moreinstructions, memory, and communication devices to transmit and receivedata over the network 120. In some embodiments, the recommendationservice 102 is implemented on one or more backend servers capable ofcommunicating over a network. In other embodiments, the recommendationservice 102 is implemented by one or more virtual machines in a hostedcomputing environment.

In one aspect, the color recommendation service 102 can correspond toone or more applications that perform, individually or in combination,the recommendation functions described herein, including recommending animage, recommending an item of a specified color, recommending itemsthat coordinate with a user specified color, recommending sets of colorcoordinated items, recommending items of selected colors based on usertextual and/or image-based search queries, etc. Recommendations of suchimages and/or items may correspond to matching or affiliated colors,palettes and/or items as will be further described herein.Recommendations may include item category recommendations, brand-relatedrecommendations, price-related recommendations, etc. In another aspect,the recommendation service 102 may be configured to identify trends initems and utilize such trends to provide recommendations. In someembodiments, the recommendation service 102 is associated with a networkor network-based merchandise provider, vendor and/or other parties.

The recommendation service 102 may be communicatively connected to thepalette data store 110 and the dictionary data store 112. The palettedata store 110 can generally include any repository, database, orinformation storage system that can store palette data and associatedmetadata. The dictionary data store 112 can generally include anyrepository, database, or information storage system that can storedictionaries, such as dictionaries that may be used to parse queries,identify keywords, identify colors, identify color palettes, ordetermine coordinating items, as discussed elsewhere herein.

The palette data stored in the palette data store 110 can be collectionsof colors optionally with associated name(s), weight(s) and date(s) ofcreation. Palette data can be of various formats, such as lists,vectors, arrays, matrices, etc. Metadata may optionally be associatedwith individual palettes for purposes of textually indicating thecolor(s) included in the palette using color name(s) or otheridentifier(s), and optionally indicating their format, semantics,features, conditions, sources, date of creation/editing, associateddemographics (e.g., geographical region, age, gender, ethnic group, etc.of users that provided input used in creating the palette), or the like.An example process of generating color palettes is discussed in greaterdetail herein.

Using an initial color or colors, an ordered list of affiliated colorscan be generated where a given affiliated color is ranked based at leastin part on the popularity of the combination of the initial color orcolors with that affiliated color. The color palette can be built byadding an affiliated color to the colors in the palette and thenupdating the list of affiliated colors to suggest new affiliated colorsto add to the updated palette. The resulting color palette can beconfigured to contain a combination of colors that is visually appealingor preferable because each affiliated color used in generating the colorpalette has been determined by the community of people to be anappropriate or preferable color companion to the color or colors alreadyin the palette. The palettes generated using the affiliated colorprocess may be used to provide color-related recommendations for colorsor colored items that would go well with another color or colored item.Particular color palettes may be associated with a particular communitythat includes a biased population (e.g., that are related based ongeographical region, age, gender, ethnic group, religion, culture,language, dialect, preferences, social network, etc.). This enablesproviding recommended colors to users that have a known and/or inferredbias that corresponds to a palette of a community associated with suchcolor palette bias.

In some embodiments, a first color can be selected by a program or auser and a plurality of palettes can be identified from a data store ofpalettes that contain that color (or a sufficiently similar color). Fromthose palettes, a list of affiliated colors can be generated byidentifying the other colors in the palettes. For each affiliated colorin the list, a weight can be assigned based on the ranking, rating,and/or number of votes the containing palette has received. The list ofaffiliated colors can be sorted based on the assigned weights. Theprogram or user can select an affiliated color from the sorted list toadd to a custom color palette containing the initial color. When theselected affiliated color is added to the palette, a new list ofaffiliated colors can be generated based at least in part on the colorsin the palette that allows the program or user to continue to build thecolor palette. For more example details on extracting colors from animage to obtain a color palette, see U.S. patent application Ser. No.14/316,483, entitled “IMAGE-BASED COLOR PALETTE GENERATION,” filed onJun. 26, 2014, and corresponding to Attorney Docket No. SEAZN.912A1,which is incorporated by reference herein in its entirety. For moreexample details on identifying affiliated colors and/or building anaffiliated color palette based at least in part on an input color orpalette, such as an input color or palette identified from a referenceimage, see U.S. patent application Ser. No. 14/315,700, entitled “FASTCOLOR SEARCHING,” filed on Jun. 26, 2014, and corresponding to AttorneyDocket No. SEAZN.911A, which is incorporated by reference herein in itsentirety.

The commerce service 106 may provide an electronic catalog to whichthird party users 140 may be provided access via respective userdevices. For example, the commerce service 106 may provide item detailpages. A given item detail page may include detailed informationregarding an item (e.g., an item being offered for sale), such as one ormore images, descriptive text, color name(s), a price, weight, sizeoptions, reviews of the item by other users or by professionalreviewers, alternative similar items, and/or other information. The itemdetail page may also include controls that the user can use to selectamong various versions of the item (e.g., size, color, etc.), and apurchase control that the user can use to initiate purchase of the item(e.g., by adding the item to a shopping cart). The commerce service 106may also provide third party users 140 with interfaces that the user canuse to request recommendations and submit queries, such as color-relatedrecommendations and search queries.

The network 120 may include any suitable combination of networkinghardware and protocols necessary to establish communications within theoperating environment 100. For example, the network 120 may includeprivate networks such as local area networks (LANs) or wide areanetworks (WANs) as well as public or private wireless networks,satellite networks, cable networks, cellular networks, or the Internet.In such an embodiment, the network 120 may include hardware (e.g.,modems, routers, switches, load balancers, proxy servers, etc.) andsoftware (e.g., protocol stacks, accounting software, firewall/securitysoftware, etc.) that establishes networking links within the colorrecommendation processing environment 100. Additionally, the network 120may implement one of various communication protocols for transmittingdata between components of the operating environment 100.

The item data store 130 may be associated with one or more network sitesand systems, such as a commerce network site providing the commerceservice 106 and the recommendation service 102, and/or third partymerchandise providers or vendors that may market items via the commerceservice 106. The item data store 130 may be associated with anycomputing device(s) that can facilitate communication with therecommendation service 102 via the network 120. Such computing devicescan generally include servers, desktops, laptops, wireless mobiledevices (e.g., smart phones, PDAs, tablets, wearable computing devices,or the like), game platforms or consoles, electronic book readers,television set-top boxes, televisions (e.g., internet TVs), andcomputerized appliances, to name a few. Further, such computing devicescan implement any type of software (such as a browser or a mobile mediaapplication) that can facilitate the communications described above.

The item data store 130 may have metadata/keywords that identify and/ordescribe the respective items. By way of example, the item data store130 may store item records for respective items in one or moreelectronic catalogs including unique item identifiers, such as UniversalProduct Codes (UPC), European Article Numbers (EAN), InternationalStandard Book Numbers (ISBN), and/or other identifiers. By way offurther example, the item metadata may indicate the item type and/orcategory, such as “dress” and “clothing,” or “blender” and “kitchenappliance.” In addition, the item metadata may include text or anothercolor identifier (sometimes referred to herein as a “color name”)identifying one or more colors of the item or of versions of the item,such as “red,” “orange,” “blue,” etc. The metadata may further includesuch information as brand. Other data, such as price, may be included asmetadata or otherwise made accessible. Still further, a given itemrecord may include one or more images of the item, where the image maybe associated with metadata (e.g., identifying items in the image byitem type, item category, unique identifier, identifying associatedcolor palettes, etc.). Certain keywords may not identify a colorexplicitly, but may be suggestive of a color (e.g., “summery,” “formal,”“wintery,” etc.). Item record data may have been provided by an operatorof a commerce or other network site, by consumers, merchandisers,vendors, third party data stores, artists, designers, color providers,and/or other sources. As used herein, the term “item,” in addition tohaving its ordinary meaning, is used interchangeably to refer to an itemitself (e.g., a particular product) and to its description orrepresentation in a computer system or electronic catalog. As will beapparent from the context in which it is used, the term is alsosometimes used herein to refer only to the item itself or only to itsrepresentation in the computer system.

Third party users 140 may correspond to visitors to a network site(e.g., a commerce network site providing commerce service 106), such asconsumers, designers, architects, or the like, and can be associatedwith any computing device(s) that can facilitate communication with therecommendation service 102 via the network 120. Such computing devicescan generally include wireless mobile devices (e.g., smart phones, PDAs,tablets, wearable computing devices, or the like), desktops, laptops,game platforms or consoles, electronic book readers, television set-topboxes, televisions (e.g., internet TVs), and computerized appliances, toname a few. Further, such computing devices can implement any type ofsoftware (such as a browser or a mobile media application) that canfacilitate the communications described above.

The color palette providers 150 can create and/or curate colorcombinations based on the preferences of each provider's community ofusers. Particular color palette providers 150 may be associated with aparticular community that includes a biased population or a populationwith unique or particular preferences. This may allow for the affiliatedcolor service 105 to retrieve palettes with a known and/or desired biasdepending at least in part on the use of the retrieved palettes. Thismay also allow for the affiliated color service 105 to reduce or removethe bias present in different communities by combining palettes from aplurality of communities of users.

The color palette providers 150 can be associated with any computingdevice(s) that can facilitate communication with the affiliated colorservice 105 via the network 120. Such computing devices can generallyinclude network servers, desktops, laptops, wireless mobile devices(e.g., smart phones, PDAs, tablets, wearable computing devices, or thelike), game platforms or consoles, electronic book readers, televisionset-top boxes, televisions (e.g., internet TVs), and computerizedappliances, to name a few. Further, such computing devices can implementany type of software (such as a network server) that can facilitate thecommunications described above.

One skilled in the relevant art will appreciate that the components andconfigurations provided in FIG. 1 are illustrative in nature.Accordingly, additional or alternative components and/or configurations,especially regarding the additional components, systems and subsystemsfor facilitating functions disclosed herein may be utilized.

FIG. 2 is a block diagram illustrating an embodiment of examplecomponents of a computing system 200 implementing at least one of arecommendation service 102, an image processing service 104, anaffiliated color service 105, or a commerce service 106, utilized inaccordance with the operating environment 100 of FIG. 1. The examplecomputing system 200 includes an arrangement of computer hardware andsoftware components that may be used to implement aspects of the presentdisclosure. Those skilled in the art will appreciate that the examplecomponents may include more (or fewer) components than those depicted inFIG. 2. It is not necessary, however, that all of these generallyconventional components be shown in order to provide an enablingdisclosure.

The computing system 200 may include a processing unit 202, a networkinterface 204, a non-transitory computer-readable medium 206, and aninput/output device interface 208, that all may communicate with oneanother by way of a communication bus. The network interface 204 mayprovide the recommendation service 102, the image processing service104, the affiliated color service 105, and/or the commerce service 106with connectivity to one or more networks or computing systems. Theprocessing unit 202 may thus receive information and instructions fromother computing devices, systems, or services via a network. Theprocessing unit 202 may also communicate to and from memory 210 andfurther provide output information via the input/output device interface208. The input/output device interface 208 may also accept input fromvarious input devices, such as a keyboard, mouse, digital pen, touchscreen, etc.

The memory 210 may contain computer program instructions that theprocessing unit 202 may execute in order to implement one or moreembodiments of the present disclosure. The memory 210 generally includesRAM, ROM and/or other persistent or non-transitory computer-readablestorage media. The memory 210 may store an operating system 214 thatprovides computer program instructions for use by the processing unit202 in the general administration and operation of the recommendationservice 102, the image processing service 104, the affiliated colorservice 105, and/or the commerce service 106. The memory 210 may furtherinclude other information for implementing aspects of the presentdisclosure.

In an example embodiment, the memory 210 includes an interface module212. The interface module 212 can be configured to facilitate generatingone or more user interfaces through which an item data store 130 or athird party user 140, utilizing a compatible computing device, may sendto, or receive from, the recommendation service 102 recommendations,image data, palette data, instruction data, metadata, etc., or otherwisecommunicate with the recommendation service 102. Specifically, theinterface module 212 can be configured to facilitate processingfunctions described herein, including recommending an image,recommending an item, recommending a keyword, recommending items thatcoordinate with a user specified item, recommend an assemblage ofcoordinating items (e.g., clothing, furniture, appliances, housewares,linen, etc.) in coordinating colors, recommending items of selectedcolors based on user textual search queries, validating metadata,processing purchase transactions, etc. Recommendations of such images,items, and/or palettes may correspond to matching or affiliated colors,palettes and/or items as will be further described herein.

For example, a third party user 140 may submit a color-relatedrecommendation query or selection and the item data store 130 mayprovide data used to satisfy the query or selection, including itemimages, item colors, item categories, item descriptions, item prices,etc. The third party user may submit queries or selections and receiverecommendations via one or more generated user interfaces. The userinterface can be implemented as a graphical user interface (GUI),Web-based user interface, computer program, smartphone or tablet programor application, touchscreen, wearable computing device interface,command line interface, gesture, voice, or text interface, etc., or anycombination thereof.

In addition, the memory 210 may include a data processing module 216that may be executed by the processing unit 202. In an exampleembodiment, the data processing module 216 implements aspects of thepresent disclosure. For example, the data processing module 216 can beconfigured to process user queries, instructions, data from the itemdata store 130, palette data from the palette data store 110, ormetadata to generate image-based recommendations.

It should be noted that the computing system 200 may include some or allof the components present in the computing system 200 as discussedherein with respect to FIG. 2. In addition, the computing system 200 mayinclude additional components not present in FIG. 2. The modules orcomponents described above may also include additional modules or beimplemented by computing devices that may not be depicted in FIG. 1 or2. For example, although the interface module 212 and the dataprocessing module 216 are identified in FIG. 2 as single modules, oneskilled in the relevant art will appreciate that each of the modules maybe implemented by two or more modules and in a distributed manner. Asanother example, the computing system 200 and its components can beimplemented by network servers, application servers, database servers,combinations of the same, or the like, configured to facilitate datatransmission to and from item data store 130, third party users 140, orother image sources, via network 120. Accordingly, the depictions of themodules are illustrative in nature.

Several example routines will now be described with reference to thefigures. It is understood that more than one of the routines or portionsthereof may be utilized to generate recommendations, keywords, or colorpalettes in response to a given user query, selection, or other input.

Example Process to Generate a List of Images Associated with an InputColor Palette

FIG. 3 is a flow diagram illustrating an embodiment of a routineimplemented by the recommendation service 102 for providingrecommendations based at least in part on a color palette. Therecommendation service 102 begins the routine at block 300. At block302, the recommendation service 102 obtains a color palette. The colorpalette can be received from a user (e.g., a third party user 140 viadata transmission to the recommendation service 102), from anothersystem, retrieved from a data store (e.g., palette data store 110), orgenerated randomly. In some embodiments, the color palette can beextracted from an image, such as an image provided by a user or an imageof an item in an item catalog or on a network site. Various systems andmethods for extracting colors from an image are described in U.S. patentapplication Ser. No. 14/316,483, entitled “IMAGE-BASED COLOR PALETTEGENERATION,” filed on Jun. 26, 2014, and corresponding to AttorneyDocket No. SEAZN.912A1, which is incorporated by reference herein in itsentirety. In some embodiments, the color palette can be identified usinga color palette name and/or other color identifier(s). The color palettecan be a single color or a plurality of colors. The color palette can berepresented using any conventional color representation including, forexample, RGB values, YUV values, color names, or the like.

In some embodiments, the color palette is received as a text query(where the text query may comprise user-entered text or a selection oftext by the user from a menu or otherwise, and the query includes acolor name). The text query may include information in addition tocolor, such as item type (e.g., dress, blender, couch, or other specificitem type), item category (e.g., clothing, appliance, furniture, etc.),color name (e.g., red, blue, orange, etc.), and/or color palette name(e.g., a color palette name as used by the color palette providers 150or as stored in the palette data store 110). The query may also includeother terms or filters (e.g., entered as text, selected from a menu, orotherwise provided), such as a price range, desired brands, etc., wherethe user is looking for recommendations of items corresponding to suchterms or filters.

At block 304, the recommendation service 102 determines one or moresearch colors from the color palette obtained at block 302. In someembodiments, the color palette can include a plurality of colorsrepresented in a color space. For example, the color palette canincluded colors extracted from an image, where the extracted colors arerepresented in the color space. As another example, the color palettecan be obtained as a vector, array, or matrix of color values in thecolor space. In some embodiments, the recommendation service 102 mayparse a text query to identify color names and translate those colornames into color values or representations of color in the color space.As used herein, a color space includes any system that allows color tobe represented using numeric values including, for example and withoutlimitation, CIE, RGB, YUV, HSL/HSV, CMYK, and the like. For example, ifthe query is “red dress,” the recommendation service 102 may use adictionary, such as dictionary data store 112, to determine that theterm “red” is a color and that the term “dress” is an item type. Therecommendation service 102 may then determine what color or range ofcolors in the color space corresponds to the term “red.”

For color comparisons, the recommendation service 102 may determine arange of colors that it will consider sufficiently similar to a searchcolor derived from the obtained color palette. The range of colors canbe determined using a color distance threshold applied in a particularcolor space. For example, the recommendation service 102 may apply acolor distance threshold according to a color distance formula(e). Anexample of such a formula is based on a human perceptible colordifference. For more information on how to determine a human perceptiblecolor difference and the human color distance formula, please see U.S.patent application Ser. No. 14/316,483, entitled “IMAGE-BASED COLORPALETTE GENERATION,” filed on Jun. 26, 2014, and corresponding toAttorney Docket No. SEAZN.912A1, which is incorporated by referenceherein in its entirety. In some embodiments, the threshold used todetermine sufficiently similar colors can be altered, tuned, or modifiedby a user or other system to broaden or narrow the search orrecommendation results.

At block 306, the recommendation service 102 identifies images thatcontain one or more of the search colors determined at block 304. Therecommendation service 102 can retrieve or receive color images foranalysis. A color image can depict one or more items (e.g., clothing,furniture, appliances, etc.), a design, a scene, a landscape, or anyother content of color. The recommendation service 102 can obtain acolor image by retrieving or receiving image data from the item datastore 130, third party users 140 (e.g., consumers or merchandisersseeking to sell items), or other image sources, via transmission ofimage data to the recommendation service 102.

The recommendation service 102 can analyze the color images to extractcolors from the image. For an example of extracting colors from an imageto obtain a color palette, see U.S. patent application Ser. No.14/316,483, entitled “IMAGE-BASED COLOR PALETTE GENERATION,” filed onJun. 26, 2014, and corresponding to Attorney Docket No. SEAZN.912A1,which is incorporated by reference herein in its entirety. Therecommendation service 102 may use fast color searching techniques, asdescribed in U.S. patent application Ser. No. 14/315,700, entitled “FASTCOLOR SEARCHING,” filed on Jun. 26, 2014, and corresponding to AttorneyDocket No. SEAZN.911A, which is incorporated by reference herein in itsentirety, to compare the search color(s) to the extracted colors. Insome embodiments, the recommendation service 102 can use a colorthreshold such that any color within the color threshold is consideredto be a match to the search color.

The recommendation service 102 can compare the search color to thecolors extracted from a color image. If one or more of the extractedcolors matches or corresponds to the search color, the color image isidentified as an image that contains the search color. Colors can beconsidered to match or correspond where the colors are sufficientlysimilar, such as where the colors are less than a threshold distanceapart in a color space. In certain embodiments, the recommendationservice 102 can identify an image as containing the search color wherethe image contains at least one color that matches the search color. Insome embodiments, the recommendation service 102 can identify an imageas containing the search color where the matching color in the imagecomprises at least a minimum percentage of the image. This may beadvantageous to avoid identifying an image as containing the searchcolor if that color appears in a small number of pixels in the image.Where the search color comprises a plurality of search colors, therecommendation service 102 can identify an image as containing thesearch colors where it includes all of the search colors, where itincludes a majority of the search colors, or where it includes at leastone of the search colors.

The recommendation service 102 can rank or order the identified imagesbased at least in part on the quality of the color matching. Forexample, where the search color matches a large portion of the image,the image can be ranked relatively high. As another example, where thesearch color matches a distinct color in the image, the image can beranked relatively high. Where there is a plurality of search colors, animage can be ranked relatively high if it contains a large fraction ofthe search colors.

At block 308, the recommendation service 102 analyzes metadataassociated with the images identified at block 306. The recommendationservice 102 may access the item data store 130, which may correspond toan electronic catalog of items, to extract metadata corresponding to theidentified images. For example, item records in the item data store 130may have tags or other metadata/keywords that identify describe therespective items. The item data store 130 may store item records forrespective items in one or more electronic catalogs including itemidentifiers. By way of further example, the item metadata may includeitem identifier(s) that indicate the item type and/or category, such as“dress” and “clothing,” or “blender” and “kitchen appliance.” Inaddition, the item metadata may include text or other metadataidentifying one or more colors of the item or of versions of the item,such as the color names “red,” “orange,” “blue,” etc. The item metadatamay include keywords, such as non-color specific keywords, associatedwith the item and/or colors included in the item. The metadata mayfurther include such information as brand. Other data, such as price,may be included as metadata or otherwise accessed. Item record data mayhave been provided by an operator of a commerce site, by consumers,third party data stores, and/or other sources.

The recommendation service 102 may compare information obtained from thetext query with item metadata or other data to identify items thatcorrespond to the query. The recommendation service 102 may store atemporary record of the matching items (e.g., by storing a list ofcorresponding item identifiers). The recommendation service 102 may rankthe matching items in accordance with the closeness of the match to thequery to provide an item relevancy ranking. For example, itemidentifiers in the text query can be used to rank matching items basedat least in part on a comparison between the item identifiers andinformation in the item metadata. In some embodiments, user profileinformation and/or demographics may be used to rank matching items. Forexample, a user's age, gender, geographical location, job, maritalstatus, or the like can be used to determine recommended items from thegrouping of matching items. In some embodiments, a user's preferencesmay be used in ranking matching items. By way of example, the user'spreferences may be determined based on the user's purchase or browsehistory (which may indicate the colors of items purchased or browsed bythe user), items is a user's wish list, prior user color-related searchqueries, or preferences explicitly provided by the user via a preferenceform or otherwise.

At block 310, the recommendation service 102 generates a filtered set ofitems based at least in part on the quality of color matching betweenthe obtained color palette and the colors in the image and the imagemetadata. In some embodiments, the recommendation service 102 utilizesthe text query to further filter out items that do not sufficientlymatch the information in the query. For example, if the text queryspecifies “blue couch,” a blue shirt will be excluded from the set ofitems. Optionally, the recommendation service 102 may rank the filteredlist according to the ranking of matching items performed at block 306.Optionally, the color recommendation service 102 may rank the filteredlist according to the ranking of closeness of the item color to thecolor query. Optionally, some combination of the ranking of matchingitems performed at block 306 and a ranking based on the closeness of theitem color to the color query may be used to generate a furtherrelevancy ranking of items.

The identified images may be ranked in accordance with a determinationas to how closely a given image corresponds to the search colors. Forexample, palettes may have an assigned weight indicating which is themost dominant color (where a color may or may not be a shade of acolor), the second most dominant color, and so on. Optionally, onlyimages that match within a percentage of the search colors, or only aspecified maximum number of images will be identified (e.g., the 10closest images). Other factors may be taken into account in weightingimages. For example, attitudes and perceptions of colors, and whatcolors coordinate with what colors, may change over time, based ongender, geographic region, ethnic group, age, culture, religion, etc.Thus, the image weighting may be based at least in part on one or moreof the image date, and optionally on one or more of the user's gender,geographic region, ethnic group, age, culture, religion, palettepopularity trends, etc.

At block 312, the recommendation service 102 provides the recommendeditems and/or images. The items and/or images can be provided for displayon a user device (e.g., via a browser installed on the user device, viaa dedicated application such as a mobile device app, or otherwise) asrecommended items based at least in part on the color palette (e.g.,which may include a user's query). The recommended items may be in theform of only images of the matching items, where the images may beretrieved from the item data store 130. The recommended items may be inthe form of only text identifying and/or describing the matching items,where the text may be retrieved from the item data store 130. Therecommended items may be in the form of both images of the matchingitems and corresponding text retrieved from the item data store 130.Optionally, controls may be provided to the user that the user can useto indicate that a larger or a smaller number of matching items are tobe presented. In response, the recommendation service 102 mayaccordingly modify the number of items presented to the user. Forexample, if the user indicates that fewer items are to be presented tothe user, the recommendation service 102 may present the higher rankeditems and not present a number of lower ranked items that had previouslybeen presented. In some embodiments, the recommendation service 102 isconfigured to generate a user interface, wherein the user interface isconfigured to display one or more of the matching images, the imagemetadata for the matching images, or a combination of both matchingimages and associated metadata.

Optionally, the recommendation service 102 may utilize metadataassociated with an item selected by the user to identify similar and/orcoordinating items to the user. For example, the recommendation service102 may utilize color information associated with the selected item(e.g., associated with the image of the selected item), the item typeinformation, and/or the item categorization information to identifyrelated and/or coordinating items. For example, if the user selected adress having a palette with a deep shade of red as the primary color,the recommendation service 102 may identify and present items having asimilar palette with deep red as a primary color and/or having acoordinating color palette (an affiliated color) different than theprimary color of the item. By way of further example, the recommendationservice 102 may identify item accessories (e.g., a scarf, earrings,handbag, etc.) whose color(s) coordinate with the selected dress. Asanother option, the recommendation service 102 may identify itemaccessories or coordinating items in response to a user query (e.g.,where the user provides or selects an additional item type or itemcategory query). In some embodiments, the recommendation service 102 canidentify similar and/or coordinating items by accessing information fromthe item data store 130, wherein information for an item in the itemdata store 130 includes one or more similar and/or coordinating items.In some embodiments, the recommendation service 102 can identify similarand/or coordinating items by accessing information about an itemidentifier in the dictionary data store 112, wherein the information forthe item identifier (e.g., dress) includes a list of different itemidentifiers that are similar or that coordinate with an item describedby the item identifier. The recommendation service 102 ends the routineat block 320.

By way of example, a merchandiser may want to design a collection ofitems that are based on a color palette. The merchandiser can provide aninput image with a desired color palette, color values in the colorpalette, the name of the color palette and/or names for the colors inthe color palette and receive a list of items that contain the colors inthat palette. The merchandiser may be able to provide item identifiersto the recommendation service 102 to rank and/or filter the returnedlist of items. This can allow the merchandiser to search and findrecommendations based on color names as well as other information in aquery.

By way of further example, a user may want to build an outfit based onthe color blue. The user may submit a query “blue outfit.” Therecommendation service 102 may parse the query and determine that “blue”is a color and “outfit” is an assemblage of clothing items, optionallyincluding accessories. Based on the queried color, routine depicted inFIG. 3 can determine, from the palette data store 110, a color palette(e.g., a group of one or more colors that correspond to the color in thequery). The recommendation service 102 may then identify clothing itemshaving the color palette, and assemble one or more outfits accordingly.Items in a given assembled outfit may be presented to the user as a setto indicate that the items in the set are part of the same recommendedoutfit.

Example Process to Validate a Color or Palette Name with Image Metadata

FIG. 4 is a flow diagram illustrating an embodiment of a routineimplemented by the image processing service 104 for validating colornames and/or information in image metadata based at least in part oninput color names. A merchandiser (or other vendor or party) may submitan image, such as a photograph, of an item for posting on an item detailpage of a commerce site that consumers may use to purchase the item.Typically, the image will be associated with metadata provided by themerchandiser, including one or more color names identifying the item (oritems) in the image. However, not infrequently, the color name submittedby the merchandiser (or other source) is in error and does not match thecolor of the item in the image. For example, an image of a blue dressmay be submitted with a color name of “purple.” Having erroneous colornames results in inaccurate image records and can result in incompleteor inaccurate recommendations. Thus, for example, if a search isperformed for a blue dress (e.g., in response to a user query and/or aspart of a recommendation process), the blue dress mistakenly tagged withthe color name “purple” may not be identified in the search or to theuser. Conversely, if a search is performed for a purple dress, the bluedress mistakenly tagged with the color name “purple” will be erroneouslyidentified in the search and to the user as part of a recommendation. Insome embodiments, a user interface may be generated to present colorvalidation information, color names, erroneous information, comparisondata, metadata associated with images, and/or images to a user.

Certain embodiments address the foregoing challenges by determining whencertain image metadata, such as a color or color palette name, does notmatch the color palette of an item in the image. As will be discussed ingreater detail below, the image processing service 104 analyzes an inputcolor name or names; determines one or more colors from the colorname(s); identifies one or more images with the determined colors;accesses metadata associated with the identified image(s), the metadataincluding names of colors in the image; compares the information in themetadata with the color names; and identifies differences between theinput color name(s) and the color names in the metadata. If the colornames from the metadata do not match or correspond to those determinedfrom the input color name(s), the image processing service 104 can editthe metadata to include the color names and/or alert an entity (such asan operator and/or the entity that provided the image) via anotification regarding the mismatch. In response to the mismatchnotification, the entity that provided the image can manually comparethe color names from the image metadata that do not match those colornames, enabling the entity to determine which color name is correct. Theentity can then select the appropriate color name to be stored asmetadata with the image, update the definition of the color name, or addthe color name to a repository of color names (e.g., the palette datastore 110). Thus, searches and recommendations related to the item willbe more accurate and complete. In some embodiments, the image processingservice 104 identifies and/or receives image(s); determines validationcolor name(s) based at least in part on color(s) from the image(s);accesses metadata associated with the image(s), the metadata includingnames of colors in the image; and identifies differences between thevalidation color name(s) and the color names in the metadata. Forexample, the image processing service 104 may automatically processimages from the item data store 130 to validate metadata associated withthe images and/or notify an entity about discrepancies with the colorinformation in the metadata. In some embodiments, the image processingservice 104 may execute as a batch process to validate colorinformation.

Similarly, if an image is provided that does not include a color name inits metadata, the image processing service 104 can process the image toextract one or more colors from the image, identify color names thatcorrespond to the extracted colors, and edit the metadata to include theidentified color names. This enables the item to be discovered in asearch for the item type or category having the color as a searchparameter. For example, the search may be performed in response to auser query (e.g., including a user provided or selected keyword and/orimage) and/or as part of a recommendation process, as describedelsewhere herein. Thus, searches and recommendations related to the itemwill be more accurate and complete.

The image processing service 104 begins the routine of FIG. 4 at block400. At block 402, the image processing service 104 analyzes a colorname, referred to as the reference color name. The reference color namecan be received from a user (e.g., a third party user 140 via datatransmission to the recommendation service 102), from another system, orgenerated randomly. In some embodiments, color names may be determinedbased at least in part on the colors retrieved or extracted from images,described in further detail below. Color names may be any type of coloridentifier, including names expressed using ASCII characters, icons, orother such data. Color names may be provided by one or more surveys suchas a human survey of color names and/or human generated data of colornames. For example, a human color survey may have included hundreds ofthousands of users to name millions of RGB colors. Furthermore, thecolor names from a color survey may be associated with ranking and/orvoting data indicating relative human preferences for color names. Forexample, out of thousands of survey participants, the color name “lime”may receive the highest votes for a particular RGB color value. Thecolor names from a human color survey may be richer and/or more accuratethan standard color naming data stores and/or data sources. For example,color names based on a color survey may include names that are nottypical color names but that are highly suggestive of color, such as“lime,” “asparagus,” or the like. The data from such a survey may bestored in the palette data store 110. A service may use fast colorsearching techniques, as described in U.S. patent application Ser. No.14/315,700, entitled “FAST COLOR SEARCHING,” filed on Jun. 26, 2014, andcorresponding to Attorney Docket No. SEAZN.911A, which is incorporatedby reference herein in its entirety, to retrieve one or more names forcolors. Additionally or alternatively, a color name associated with acolor may be determined based at least in part on color ranges within acolor space, as described in further detail herein.

In some embodiments, color names may be associated with metadata and/oradditional data may be determined to be associated with color names. Forexample, a color with the name “manatee” may be associated with a textdescription such as “animal,” “sea animal,” “mammal,” “exotic animal,”or the like. In some embodiments, data associated with color names maybe determined based at least in part on natural language processing orother techniques. As a result, validation of color names associated withan image may include filtering of color names based at least in part onmetadata. For example, during color name validation, color namesassociated with animals and/or exotic animals may be excluded if usedwith images in a clothing context.

At block 404, the image processing service 104 determines a color fromthe color reference name, referred to as the reference color.Determining the reference color from the reference color name includesassociating the reference color name (e.g., from a text-based query)with a color value or range of values in a color space. The imageprocessing service 104 can search for the reference color name in thepalette data store 110, the dictionary data store 112 and/or in anothercolor name service such as the color palette providers 150. Thereference color name can be associated with a single color, a range ofcolors, or a color palette (e.g., a group of one or more colorsassociated with a single name).

The reference color can be represented by a value or a range of valuesin a color space. In some embodiments, the image processing service 104can determine a threshold color distance such that any color that has adistance less than the threshold from the reference color is consideredto be sufficiently similar to the reference color. In certainembodiments, the threshold color distance is based at least in part onhuman perceptible color differences. This allows the image processingservice 104 to determine relatively large or relatively small areaswithin the color space for matching colors with color names. This may beadvantageous, for example, where the reference color name suggests arelatively broad range of colors (e.g., “red,” “green,” “yellow,” etc.)or a relatively narrow range of colors (e.g., “bright cerulean,”“periwinkle,” “sunset orange,” etc.).

At block 406, the image processing service 104 identifies imagescontaining the reference color determined at block 404. The imageprocessing service 104 can retrieve or receive color images foranalysis. A color image can depict one or more items (e.g., clothing,furniture, appliances, etc.), a design, a scene, a landscape, or anyother content of color. The image processing service 104 can obtain acolor image by retrieving or receiving image data from the item datastore 130, third party users 140 (e.g., consumers or merchandisersseeking to sell items), or other image sources, via transmission ofimage data to the image processing service 104.

The image processing service 104 can analyze the color images to extractcolors from the image. For an example of extracting colors from an imageto obtain a color palette, see U.S. patent application Ser. No.14/316,483, entitled “IMAGE-BASED COLOR PALETTE GENERATION,” filed onJun. 26, 2014, and corresponding to Attorney Docket No. SEAZN.912A1,which is incorporated by reference herein in its entirety. The imageprocessing service 104 may use fast color searching techniques, asdescribed in U.S. patent application Ser. No. 14/315,700, entitled “FASTCOLOR SEARCHING,” filed on Jun. 26, 2014, and corresponding to AttorneyDocket No. SEAZN.911A, which is incorporated by reference herein in itsentirety, to compare the reference color to the extracted colors. Insome embodiments, the image processing service 104 can determine a colorthreshold such that any color within the color threshold is consideredto be a match to the reference color.

The image processing service 104 can compare the reference color to thecolors extracted from a color image. If one or more of the extractedcolors matches the reference color, the color image is identified as animage that contains the reference color. Colors can be considered tomatch where the colors are sufficiently similar, such as where thecolors are less than a threshold distance apart in a color space. Incertain embodiments, the image processing service 104 can identify animage as containing the reference color where the image contains atleast one color that matches the reference color. In some embodiments,the image processing service 104 can identify an image as containing thereference color where the matching color in the image comprises at leasta minimum percentage of the image. This may be advantageous to avoididentifying an image as containing the reference color if that colorappears in a small number of pixels in the image. Where the referencecolor is a color palette comprising a plurality of colors, the imageprocessing service 104 can identify an image as containing the referencecolor where it includes all of the colors in the reference colorpalette, where it includes a majority of the colors in the referencecolor palette, or where it includes at least one color in the referencecolor palette.

At block 408, the image processing service 104 analyzes metadataassociated with each image identified at block 406. The image processingservice 104 retrieves or receives metadata from the item data store 130,for example. The metadata may include information corresponding to thecolor palette (e.g., color names), color scheme, lighting source,lighting direction, or other factors regarding the color rendering ofthe image. The metadata may also include information about the currentlyobtained color image, other color images, subjects or category ofsubjects depicted, sources contributing to the image, or theirinterrelations. The metadata can further include any other informationassociated with the color image as can be envisioned by a person ofskill in the art.

The image processing service 104 analyzes the metadata to determinewhether it contains one or more color names. The image processingservice 104 can compare the information contained in the metadata withthe dictionary data store 112, the palette data store 110 and/or othercolor name services to identify any color names in the metadata. Theimage processing service 104 can then compare any identified color namesfrom the metadata with the colors extracted from the associated image tocreate a map of metadata color names to colors extracted from the colorimage. In some embodiments, where the metadata contains fewer colornames than extracted colors, the image processing service 104 can orderthe extracted colors based on prominence in the image and match thoseprominent colors with the color names.

At block 410, the image processing service 104 compares any color namesidentified within the metadata to the reference color name to determinewhether the metadata color name(s) match the reference color name. Insome embodiments, matching of the reference color name to the colorname(s) in the metadata may be based on partial text string matching,matching of strings and/or words, fuzzy matching, natural languageprocessing, or the like and/or some combination thereof. If the imageprocessing service 104 determines that the metadata associated with theimage does not include color identification information, such as a colorname, then the reference color name may be added to the metadata of theimage (e.g., in item data store 130, a dedicated image data store, orelsewhere).

In some embodiments, the image processing service 104 can determine thatcolor names match if the colors represented by the respective colornames are sufficiently similar so as to be considered a match.Sufficient similarity can be based at least in part on color distances.For example, where the reference color name is “blue” and the metadataincludes the color name “cobalt,” the image processing service 104 canconsider this a match because the color corresponding to the color name“cobalt” lies within a color distance threshold from the colorcorresponding to the color name “blue.” In another example, the lightingconditions for an image may be such that the one or more colorsextracted from the image may not match the color metadata associatedwith the image (even though the color metadata may be correct). Thus,similar to the above, the image processing service 104 may determinethat the item's color metadata is correct because the color distancebetween the two colors may be within a threshold. In some embodiments,the color distance between the extracted color and the colorcorresponding to the metadata may be presented to a user for review.

At block 412, the image processing service 104 identifies differencesbetween the color name and the name of the color in the metadata. If theimage processing service 104 determines that the metadata color namematches the reference color name, the image processing service 104completes the routine at block 414. If the image processing service 104determines that the metadata color name does not sufficiently match thereference color name, then the reference color name may be added to themetadata of the image and optionally the metadata color name (or otheridentifier) is deleted or an indication is stored that the originalcolor name is mismatched or incorrect. If the image processing service104 determines that the metadata color name does not sufficiently matchthe reference color name, the image processing service 104 can suggest amodification of the metadata to be approved and/or executed by anothersystem or user. If the image processing service 104 determines that themetadata color name does match any known color name and the associatedcolor is sufficiently different from any color associated with knowncolor names, the image processing service 104 can add the color name tothe palette data store 110, the dictionary data store 112 and/or anothercolor name service. In this way, the data store of color names can beupdated to include new color names. The routine of FIG. 4 ends at block414.

Example Process to Identify Keywords from a Color or Palette Name

FIG. 5 is a flow diagram illustrating an embodiment of a routineimplemented by the recommendation service 102 for determining keywordsbased at least in part on input color names. The routine can be used toprovide suggested keywords that are non-color specific (e.g., summery,warm, Christmas, etc.) based on one or more input color names. In thisway, color names can be associated with non-color specific keywords thatmay allow a user, service, or system to identify items that contain adesired color, complementary colors and/or colors with a similar look orfeel.

A user may want to search for items that have a particular look or feel,on a commerce network site for example, and may begin the search byidentifying a color that the user associates with that look or feel.Searching for items with that color may result in items that contain thesearched color, but the results may exclude other items that wouldsatisfy the user because those items do not contain the searched color.For example, if the user searches for blue sweaters, the user may alsobe interested in grey sweaters because the user is searching for a“wintery” look. However, the search for blue sweaters typically willexclude grey sweaters.

A merchandiser (or other vendor) may be interested in providing avariety of keywords in the description of an item, for example usingmetadata associated with an image of the item. The merchandiser may beable to describe the colors of the item, but limiting the description tothose colors may be too narrow. If the description is too narrow, usersmay not find the items provided by the merchandiser because they areusing keywords that the merchandiser did not include in the itemdescription. For example, the merchandiser may be selling a serving bowlthat is red and orange, but a user searching for a “fiery” serving bowlmay not find the merchandiser's item due to the search terms used.

Certain embodiments address the foregoing challenges by determiningappropriate keywords from one or more input color names. As will bediscussed in greater detail, the recommendation service 104 analyzes aninput color name or names; determines one or more colors from the colorname(s); identifies one or more keywords associated with the determinedcolors; and provides those keywords. In some embodiments, therecommendation service 102 can receive suggestions for keywords toassociate with color names. In certain embodiments, the recommendationservice 102 can associate keywords with additional colors if thosecolors sufficiently similar to colors already associated with thekeywords.

With reference to FIG. 5, the recommendation service 102 begins theroutine at block 500. At block 502, where the recommendation service 102analyzes a color name, referred to as the reference color name. This issimilar to block 402 in the routine depicted and described in greaterdetail herein with reference to FIG. 4. The reference color name can bereceived from a user (e.g., a third party user 140 via data transmissionto the recommendation service 102), from another system, or generatedrandomly. As described herein, color names may be any type of coloridentifier and may be provided by one or more surveys such as a humansurvey of color names and/or human generated data of color names. Therecommendation service 102 may use fast color searching techniques, asdescribed in U.S. patent application Ser. No. 14/315,700, entitled “FASTCOLOR SEARCHING,” filed on Jun. 26, 2014, and corresponding to AttorneyDocket No. SEAZN.911A, which is incorporated by reference herein in itsentirety, to retrieve one or more names for colors. Additionally oralternatively, a color name associated with a color may be determinedbased at least in part on color ranges within a color space, asdescribed in further detail herein. In some embodiments, color names maybe associated with metadata and/or additional data may be determined tobe associated with color names. In some embodiments, data associatedwith color names may be determined based at least in part on naturallanguage processing or other techniques. As a result, color namesassociated with a keyword may include filtering of color names based atleast in part on metadata. For example, color names associated with heatmay be excluded from keywords that generally are associated withcoldness.

At block 504, the recommendation service 102 determines a color from thecolor reference name, referred to as the reference color. This issimilar to block 404 in the routine depicted and described in greaterdetail herein with reference to FIG. 4. Determining the reference colorfrom the reference color name includes associating the reference colorname with a color value or range of values in a color space. Therecommendation service 102 can search for the reference color name inthe palette data store 110, the dictionary data store 112 and/or inanother color name service such as the color palette providers 150. Thereference color name can be associated with a single color, a range ofcolors, or a color palette.

The reference color can be represented by a value or a range of valuesin a color space. In some embodiments, the recommendation service 102can determine a threshold color distance such that any color that has adistance less than the threshold from the reference color is consideredto be sufficiently similar to the reference color. In certainembodiments, the threshold color distance is based at least in part onhuman perceptible color differences. This allows the recommendationservice 102 to determine relatively large or relatively small areaswithin the color space for matching colors with color names. This may beadvantageous, for example, where the reference color name suggests arelatively broad range of colors (e.g., “red,” “green,” “yellow,” etc.)or a relatively narrow range of colors (e.g., “bright cerulean,”“periwinkle,” “sunset orange,” etc.).

At block 506, the recommendation service 102 identifies keywordsassociated with the reference color. A keyword can includenon-color-specific words that may be suggestive of one or more colors(e.g., summery, sunny, mellow, dressy, holiday, Halloween, Christmas,Chanukah, sports team name, etc.). A keyword can include words thatappear to be color-specific that are amenable to different meanings. Forexample, the word “blue” may refer to the color blue or the feeling ofmelancholy or depression. As another example, the word “orange” mayrefer to the color orange or the fruit of the same name. In suchinstances, the recommendation service 102 can be configured todisambiguate the potential meanings of the keywords. For example, whereblue is used as a keyword, it may be interpreted as being associatedwith an emotion; and where it is used as a color name, it may beinterpreted as the color. Keywords may also be combinations ofcolor-specific and non-color-specific words. The keywords can be storedin the palette data store 110, the dictionary data store 112, the itemdata store 130, and/or with a keyword service. Each keyword can includeone or more colors associated with that keyword. For example, a keywordmay include metadata, the metadata including a list of colors or colornames associated with the keyword. As another example, a data store canassociate a keyword with one or more colors or color names in the datastore. The colors can be represented using color names or as values in acolor space. The colors associated with each keyword can also includeranges of colors and/or color palettes. By way of example, the keyword“Halloween” may be associated with orange and green palettes, which aretypically associated with the Halloween holiday. By way of furtherexample, a given sports team may be associated with palettes in theteam's color. By way of yet further example, the keyword “formal” may beassociated with palettes considered more formal, such as blacks, greys,dark blues, etc.

The recommendation service 102 uses the reference color to identify oneor more keywords by comparing the reference color to the color(s)associated with the keywords. If the reference color is the same orsufficiently similar to one or more of the associated keyword colors,the recommendation service 102 identifies the keyword as correspondingto the reference color name. In some embodiments, the recommendationservice 102 can use a color distance threshold when comparing colorssuch that colors are considered sufficiently similar when the distancebetween two colors in a color space is less than the color distancethreshold. In certain embodiments, the color distance threshold isconfigurable to match colors that are relatively far apart or to matchcolors that are relatively close together in a color space.

In some embodiments, keywords are associated with one or more colornames. The recommendation service 102 can determine a color for eachcolor name associated with the keyword. The recommendation service 102can then compare the reference color to the colors determined from thekeyword color names, as described elsewhere herein. In some embodiments,keywords are associated with color palettes. The recommendation service102 can determine each color in the color palette and then compare thereference color to the colors of the keyword color palette. Therecommendation service 102 can identify the keyword as being associatedwith the reference color where the reference color matches at least onecolor in the associated color palette. In some embodiments, there can bea plurality of reference colors, such as with a color palette. In suchembodiments, the recommendation service 102 can identify a keyword asbeing associated with the reference color when at least one colorassociated with the keyword matches at least one reference color. Wherethe keyword is associated with a plurality of colors, the recommendationservice 102 can identify the keyword as being associated with thereference colors where all of the reference colors match a color in thekeyword color palette, where a plurality of reference colors matchcolors in the keyword color palette, or where at least one referencecolor matches a color in the keyword color palette. Similarly, therecommendation service 102 can identify the keyword as being associatedwith the reference colors when each color in the keyword color palettematches a reference color, when a plurality of colors in the keywordcolor palette matches reference colors, or where at least one color inthe keyword color palette matches a reference color.

At block 508, the recommendation service 102 generates a list orgrouping of keywords associated with the reference color name. Where therecommendation service 102 determines that more than one keyword isassociated with the reference color, the recommendation service 102 canorder the keywords. In some embodiments, the order of the keywords candepend at least in part on a weight associated with the keywords. Thekeywords can be weighted based at least in part on frequency of use,community votes or ranks, popularity, or the like. The recommendationservice 102 can order the keywords such that the highest weighted, mostpopular, and/or highest ranked keywords are listed first.

In some embodiments, the recommendation service 102 orders the keywordsbased on weights of the colors associated with the keywords. Forexample, colors associated with keywords can be weighted or otherwiseranked for that particular keyword. A keyword such as “beach” mayinclude colors such as blues and browns (e.g., water and sand) that havea higher weight than colors such as yellows (e.g., sun, clothes, toys,etc.). The recommendation service 102 can rank a first keyword higherthan a second keyword where the weight of the color matching thereference color is higher for the first keyword than for the secondkeyword.

In some embodiments, the recommendation service 102 orders the keywordsbased on weights of the reference colors used to identify associatedkeywords. Where there are multiple reference colors, each can have aweight assigned to it. The recommendation service can then weight andorder the keywords based at least in part on the weights of thesereference colors. For example, where a first keyword is associated witha color that matches a first reference color and a second keyword isassociated with a color that matches a second reference color, therecommendation service 102 can order the first keyword before the secondkeyword if the first reference color has a higher weight than the secondreference color. Where the keyword is associated with more than onereference color, the weights of those reference colors can be combinedto determine the weight of the keyword.

At block 510, the color recommendation service 102 ends the routine.

Optionally, the ranked list of keywords is provided for display on auser device (e.g., via a browser installed on the user device, via adedicated application such as a mobile device app, or otherwise) asrecommended keywords based at least in part on a user's query containingone or more reference color names. In some embodiments, when a user usesa color name in a search, the list of keywords is automaticallydetermined and the search results are provided based on the referencecolor name(s) and the list of automatically generated keywords. Theprovided list may be in the form of only keywords or as images of itemsthat are associated with the identified keywords, where the images maybe retrieved from the item data store 130. The provided list may be inthe form of only text identifying and/or describing items matching theautomatically generated keywords, where the text may be retrieved fromthe item data store 130. The provided list may be in the form of bothimages of the matching items and corresponding text retrieved from theitem data store 130. As yet another option, controls may be provided tothe user that the user can use to indicate that a larger or a smallernumber of matching items are to be presented. In response, therecommendation service 102 may accordingly modify the number of itemspresented to the user. For example, if the user indicates that feweritems are to be presented to the user, the recommendation service 102may present the higher ranked items and not present a number of lowerranked items that had previously been presented. In some embodiments,the recommendation service 102 is configured to generate a userinterface, wherein the user interface is configured to display one ormore of the keywords and/or the keyword colors.

Example Process to Generate a Custom Color Palette from Color Names

FIG. 6 is a flow diagram illustrating an embodiment of a routineimplemented by the recommendation service 102 for generating a customcolor palette based at least in part on a reference color name. Amerchandiser (or other vendor) may be interested in identifying colorsthat complement or go well with a particular color or colors. The listof colors that go well with another color can be built by determiningcolors that are affiliated with the reference colors. Affiliated colorscan include colors that are part of a group of colors that are ranked orrated by a community of users. Using the input of the community, curatedcolor palettes can be provided and utilized to generate popular orappealing color palettes. Determining color palettes affiliated with thereference color can allow the merchandiser to provide or design itemsand/or collections of items that utilize a popular or appealing colorpalette.

The recommendation service 102 begins the routine at block 600. At block602, the recommendation service 102 analyzes a color name, referred toas the reference color name. This is similar to block 402 in the routinedepicted and described in greater detail herein with reference to FIG.4. The reference color name can be received from a user (e.g., a thirdparty user 140 via data transmission to the recommendation service 102),from another system, or generated randomly. As described herein, colornames may be any type of color identifier and may be provided by one ormore surveys such as a human survey of color names and/or humangenerated data of color names. The recommendation service 102 may usefast color searching techniques, as described in U.S. patent applicationSer. No. 14/315,700, entitled “FAST COLOR SEARCHING,” filed on Jun. 26,2014, and corresponding to Attorney Docket No. SEAZN.911A, which isincorporated by reference herein in its entirety, to retrieve one ormore names for colors. Additionally or alternatively, a color nameassociated with a color may be determined based at least in part oncolor ranges within a color space, as described in further detailherein. In some embodiments, color names may be associated with metadataand/or additional data may be determined to be associated with colornames. In some embodiments, data associated with color names may bedetermined based at least in part on natural language processing orother techniques. As a result, color names associated with a keyword mayinclude filtering of color names based at least in part on metadata. Forexample, color names associated with heat may be excluded from keywordsthat generally are associated with coldness.

The recommendation service 102 may obtain or determine a custom palettecriterion for building the custom color palette. As described earlier,the criterion may represent a bias or preference in connection with alanguage, country, geographic region, ethnic group, social community,gender, age, time, trend, or the like. Alternatively or in addition, thecriterion may correspond to an item, category of items, service,category of services, design, category of designs, or the like. Thecriterion can be obtained or determined based on an action of a user,metadata associated with a color image, metadata associated with asubject depicted by a color image, or any other information or actionsrelated to the custom palette generation process. For example, thecriterion can be derived, at least in part, from an attribute associatedwith the reference color name. As described earlier, illustratively, thecolor name is expressed in a particular language and/or in a particularcontext, or the color name is associated with other metadata. Theparticular language, context or other metadata may facilitate adetermination of a color-related bias associated with a user or acategory of items in which the user is interested.

In some embodiments, the recommendation service 102 determines one ormore secondary color names that are related to the reference color name.The one or more secondary color names may facilitate color determinationby expanding the basis for the determination. The secondary color namemay be obtained based on groupings of similar or related color names viaa natural language processing (NLP) model. At block 604, therecommendation service 102 determines a color from the color referencename, referred to as the reference color. This is similar to block 404in the routine depicted and described in greater detail herein withreference to FIG. 4. Determining the reference color from the referencecolor name includes associating the reference color name with a colorvalue or range of values in a color space. The recommendation service102 can search for the reference color name in the palette data store110, the dictionary data store 112 and/or in another color name servicesuch as the color palette providers 150. The reference color name can beassociated with a single color, a range of colors, or a color palette.In the embodiments where one or more secondary color names areavailable, the reference color can be determined based on either thereference or secondary color names, or their combination.

The reference color can be represented by a value or a range of valuesin a color space. In some embodiments, the recommendation service 102can determine a threshold color distance such that any color that has adistance less than the threshold from the reference color is consideredto be sufficiently similar to the reference color. In certainembodiments, the threshold color distance is based at least in part onhuman perceptible color differences. This allows the recommendationservice 102 to determine relatively large or relatively small areaswithin the color space for matching colors with color names. This may beadvantageous, for example, where the reference color name suggests arelatively broad range of colors (e.g., “red,” “green,” “yellow,” etc.)or a relatively narrow range of colors (e.g., “bright cerulean,”“periwinkle,” “sunset orange,” etc.). As described earlier, the custompalette criterion may facilitate or control the determination of thereference color. For example, a color-naming bias associated with ageographic region may require filtering of data retrieved from thepalette data store 110, the dictionary data store 112 and/or anothercolor name service such as the color palette providers 150 to includeonly mappings between color names and colors that are derived from aparticular geographic region.

In some embodiments, the reference color is determined or derived fromone or more color images related to the reference color name (orcorresponding secondary color names.) As described earlier, metadataassociated with a color image may indicate one or more color names,validated or not, that correspond to colors depicted in the color image.Given the reference color name (or corresponding secondary color names),one or more color images can be identified based on a match orcorrespondence between the reference/secondary color names and the colornames included in color image metadata. As described earlier, the matchof color names may be based on partial text string matching, matching ofstrings and/or words, fuzzy matching, natural language processing, orthe like and/or some combination thereof.

Once the one or more color images are identified, representative colorscan be extracted therefrom. Various systems and methods for extractingcolors from an image are described in U.S. patent application Ser. No.14/316,483, entitled “IMAGE-BASED COLOR PALETTE GENERATION,” filed onJun. 26, 2014, and corresponding to Attorney Docket No. SEAZN.912A1,which is incorporated by reference herein in its entirety. Alternativelyor in addition, the color image metadata may also indicate region(s) inthe color image that correspond to a color name included in themetadata. For example, the metadata may indicate that a portion of theforeground of the color image corresponds to the color name. In thiscase, a corresponding reference color can be determined based on theindicated region(s), for example, by averaging RGB color valuescorresponding to each pixel within the regions. In some embodiments,additional color images can be identified to expand the basis fordetermining reference colors. For example, an identified color imagewith matching color names in its metadata may be associated with anexisting color palette, which is also associated with one or moreadditional color images. These additional color images may not haveappropriate metadata for direct color name matching, yet they may stillbe identified as a basis for determining reference colors.

Similarly, the custom palette criterion may facilitate or control theidentification of color images. For example, the criterion may indicatethat the user is only interested in a category of items, such asapparel. Corresponding filtering can thus be applied to the color imageidentification process such that only images depicting apparel items areselected. As described earlier, the filtering can be based on metadataassociated with color images.

At block 606, the recommendation service 102 performs the affiliatedcolor routine depicted and described in greater detail herein withreference to FIG. 8, to generate an ordered grouping or list ofaffiliated colors. The ordered list of affiliated colors contains colorsthat have been determined to go well with the reference color by acommunity of users. In some embodiments, at block 606 the recommendationservice 102 performs an affiliated color palette routine described ingreater detail herein with reference to FIG. 10, to generate anaffiliated color palette rather than just a list of ordered affiliatedcolors. The affiliated color palette routine may be an extension of anaffiliated color routine depicted in FIG. 8 that allows a user, system,or service to generate a palette of colors affiliated with the referencecolor name(s) by iteratively selecting an affiliated color from theordered list of affiliated colors provided by the affiliated routinedepicted in FIG. 8. Similarly, the custom palette criterion mayfacilitate or control the affiliated color routine depicted in FIG. 8,for example, by indicating a bias or preference for assigning oradjusting weights to colors or indicating a particular category of colorpalettes that may serve as basis for determining affiliated colors.

The routine depicted in FIG. 6 can be used by a merchandiser (or othervendor) to generate appealing color palettes for their items. Themerchandiser can enter in a color name (e.g., “teal”) and build apalette of colors affiliated with the color name. The merchandiser canbuild the palette using colors determined by a community of users to gowell with the initial named color. The merchandiser can then createcollections based on one or more affiliated color palettes to increasethe desirability of the items or item line.

Example Process to Generate an Affiliated Color Palette Using ThresholdValues Associated with Color Names

FIG. 7 is a flow diagram illustrating an embodiment of a routineimplemented by the recommendation service for generating an affiliatedcolor palette based at least in part on colors determined using areference color name and a color threshold value. The routine can beused as a variation of the routine depicted in FIG. 6 to configure thesensitivity of the affiliated color routine to identify more or feweraffiliated colors based on the input color name. In addition, routinefurther identifies images and/or metadata that include the affiliatedcolors. In this way, for example, the routine depicted in FIG. 7 allowsa user to input a color name and receive in return images and/ordescriptions of items that contain colors affiliated with the inputcolor.

The recommendation service begins the routine at a block 700. At block702, the recommendation service 102 analyzes a color name, referred toas the reference color name. This is similar to block 402 in routinedepicted and described in greater detail herein with reference to FIG.4. The reference color name can be received from a user (e.g., a thirdparty user 140 via data transmission to the recommendation service 102),from another system, or generated randomly. As described herein, colornames may be any type of color identifier and may be provided by one ormore surveys such as a human survey of color names and/or humangenerated data of color names. The recommendation service 102 may usefast color searching techniques, as described in U.S. patent applicationSer. No. 14/315,700, entitled “FAST COLOR SEARCHING,” filed on Jun. 26,2014, and corresponding to Attorney Docket No. SEAZN.911A, which isincorporated by reference herein in its entirety, to retrieve one ormore names for colors. Additionally or alternatively, a color nameassociated with a color may be determined based at least in part oncolor ranges within a color space, as described in further detailherein. In some embodiments, color names may be associated with metadataand/or additional data may be determined to be associated with colornames. In some embodiments, data associated with color names may bedetermined based at least in part on natural language processing orother techniques. As a result, color names associated with a keyword mayinclude filtering of color names based at least in part on metadata. Forexample, color names associated with heat may be excluded from keywordsthat generally are associated with coldness.

At block 704, the recommendation service 102 determines a color from thecolor reference name, referred to as the reference color. This issimilar to block 404 in the routine depicted and described in greaterdetail herein with reference to FIG. 4. Determining the reference colorfrom the reference color name includes associating the reference colorname with a color value or range of values in a color space. Therecommendation service 102 can search for the reference color name inthe palette data store 110, the dictionary data store 112 and/or inanother color name service such as the color palette providers 150. Thereference color name can be associated with a single color, a range ofcolors, or a color palette. The reference color can be represented by avalue or a range of values in a color space.

At block 706, the recommendation service 102 determines a thresholdcolor distance such that any color that has a distance less than thethreshold from the reference color is considered to be sufficientlysimilar to the reference color. In certain embodiments, the thresholdcolor distance is based at least in part on human perceptible colordifferences. This allows the recommendation service 102 to determinerelatively large or relatively small areas within the color space formatching colors with color names. This may be advantageous, for example,where the reference color name suggests a relatively broad range ofcolors (e.g., “red,” “green,” “yellow,” etc.) or a relatively narrowrange of colors (e.g., “bright cerulean,” “periwinkle,” “sunset orange,”etc.). For more example details on color distance thresholds, see U.S.patent application Ser. No. 14/316,483, entitled “IMAGE-BASED COLORPALETTE GENERATION,” filed on Jun. 26, 2014, and corresponding toAttorney Docket No. SEAZN.912A1, which is incorporated by referenceherein in its entirety.

Each reference color can be associated with a threshold. Therecommendation service 102 considers a color to be a match to thereference color if a distance in a color space between the two colorswithin the associated threshold.

At block 708, the recommendation service 102 performs the affiliatedcolor routine depicted and described in greater detail herein withreference to FIG. 8, to generate an ordered grouping or list ofaffiliated colors. As the input color to the routine depicted in FIG. 8,the recommendation service 102 uses the reference color and itsassociated color distance threshold. The ordered list of affiliatedcolors contains colors that have been determined by a community of usersto go well with the reference color and the colors within the colordistance threshold of the reference color. In some embodiments, at block708 the recommendation service 102 performs the affiliated color paletteroutine depicted and described in greater detail herein with referenceto FIG. 10, to generate an affiliated color palette rather than just alist of ordered affiliated colors. As the input color to the routinedepicted in FIG. 8, the recommendation service 102 uses the referencecolor and its associated color distance threshold. The affiliated colorpalette routine depicted in FIG. 10 is an extension of the affiliatedcolor routine depicted in FIG. 8 that allows a user, system, or serviceto generate a palette of colors affiliated with the reference colorname(s) by iteratively selecting an affiliated color from the orderedlist of affiliated colors provided by the affiliated color routinedepicted in FIG. 8.

At block 710, the recommendation service 102 identifies images thatcontain one or more of the affiliated colors determined at block 708.The recommendation service 102 can retrieve or receive color images foranalysis. A color image can depict one or more items (e.g., clothing,furniture, appliances, etc.), a design, a scene, a landscape, or anyother content of color. The recommendation service 102 can obtain acolor image by retrieving or receiving image data from the item datastore 130, third party users 140 (e.g., consumers or merchandisersseeking to sell items), or other image sources, via transmission ofimage data to the recommendation service 102.

The recommendation service 102 can analyze the color images to extractcolors from the image. For an example of extracting colors from an imageto obtain a color palette, see U.S. patent application Ser. No.14/316,483, entitled “IMAGE-BASED COLOR PALETTE GENERATION,” filed onJun. 26, 2014, and corresponding to Attorney Docket No. SEAZN.912A1,which is incorporated by reference herein in its entirety. Therecommendation service 102 may use fast color searching techniques, asdescribed in U.S. patent application Ser. No. 14/315,700, entitled “FASTCOLOR SEARCHING,” filed on Jun. 26, 2014, and corresponding to AttorneyDocket No. SEAZN.911A, which is incorporated by reference herein in itsentirety, to compare the reference color to the extracted colors. Therecommendation service 102 can use the color threshold determined atblock 706 to identify images having a color within the color threshold.

The recommendation service 102 can compare the affiliated colors to thecolors extracted from a color image. If one or more of the extractedcolors matches one or more of the affiliated colors, the color image isidentified as an image that contains an affiliated color. Colors can beconsidered to match where the colors are sufficiently similar, such aswhere the colors are less than a threshold distance apart in a colorspace. In certain embodiments, the recommendation service 102 canidentify an image as containing an affiliated color where the imagecontains at least one color that matches at least one affiliated color.In some embodiments, the recommendation service 102 can identify animage as containing an affiliated color where the matching color in theimage comprises at least a minimum percentage of the image. This may beadvantageous to avoid identifying an image as containing an affiliatedcolor if that color appears in a small number of pixels in the image. Insome embodiments, the recommendation service 102 can identify an imageas containing the affiliated colors where it includes all of the colorsin the affiliated color palette, where it includes a majority of thecolors in the affiliated color palette, or where it includes at leastone color in the affiliated color palette.

The recommendation service 102 can rank or order the identified imagesbased at least in part on the quality of the color matching. Forexample, where an affiliated color matches a large portion of the image,the image can be ranked relatively high. As another example, where anaffiliated color matches a distinct color in the image, the image can beranked relatively high. An image can be ranked relatively high if itcontains a large fraction of the colors in the affiliated color palette.

At block 712, the recommendation service 102 analyzes metadataassociated with the images identified at block 710. The recommendationservice 102 may rank the matching items in accordance with the closenessof the match to the reference color(s) to provide an item relevancyranking. Optionally, in addition, a user's preferences may be used inranking matching items. By way of example, the user's preferences may bedetermined based on the user's purchase history (which may indicate thecolors of items purchased by the user), items is a user's wish list,prior user color-related search queries, or preferences explicitlyprovided by the user via a preference form or otherwise.

In some embodiments, the recommendation service 102 generates a filteredset of items based at least in part on the quality of color matchingbetween the affiliated color palette and the colors in the image and theimage metadata. In some embodiments, the recommendation service 102utilizes a text query to further filter out items that do notsufficiently match the information in the query. For example, if thetext query specifies “blue couch,” a blue shirt will be excluded fromthe set of items. Optionally, the recommendation service 102 may rankthe filtered list according to the ranking of matching items performedat block 710. Optionally, the color recommendation service 102 may rankthe filtered list according to the ranking of closeness of the itemcolor to the affiliated color palette. As yet another option, somecombination of the ranking of matching items performed at block 710 anda ranking based on the closeness of the item color to the affiliatedcolor palette may be used to generate a further relevancy ranking ofitems.

The identified images may be ranked in accordance with a determinationas to how closely a given image corresponds to the affiliated colorpalette. For example, an affiliated color palette may have an assignedweight indicating which is the most dominant color (where a color may ormay not be a shade of a color), the second most dominant color, and soon. Optionally, only images that match within a percentage of theaffiliated colors, or only a specified maximum number of images will beidentified (e.g., the 10 closest images). Other factors may be takeninto account in weighting images. For example, attitudes and perceptionsof colors, and what colors coordinate with what colors, may change overtime, based on gender, geographic region, ethnic group, age, etc. Thus,the image weighting may be based at least in part on one or more of theimage date, and optionally on one or more of the user's gender,geographic region, ethnic group, age, culture, religion, language,palette popularity trends, etc.

In some embodiments, the recommendation service 102 provides therecommended items and/or images. The items and/or images can be providedfor display on a user device (e.g., via a browser installed on the userdevice, via a dedicated application such as a mobile device app, orotherwise) as recommended items based at least in part on the colorpalette (e.g., which may include a user's query). The recommended itemsmay be in the form of only images of the matching items, where theimages may be retrieved from the item data store 130. The recommendeditems may be in the form of only text identifying and/or describing thematching items, where the text may be retrieved from the item data store130. The recommended items may be in the form of both images of thematching items and corresponding text retrieved from the item data store130. Optionally, controls may be provided to the user that the user canuse to indicate that a larger or a smaller number of matching items areto be presented. In response, the recommendation service 102 mayaccordingly modify the number of items presented to the user. Forexample, if the user indicates that fewer items are to be presented tothe user, the recommendation service 102 may present the higher rankeditems and not present a number of lower ranked items that had previouslybeen presented. The recommendation service 102 ends the routine at block714.

Example Process to Generate a List of Affiliated Colors

FIG. 8 illustrates a flow diagram of an example routine implemented bythe affiliated color service 105 for generating a weighted or orderedlist of affiliated colors. The affiliated color service 015 begins theroutine at block 800. At block 802, the affiliated color service 105analyzes an input color. The input color can be received from a user(e.g., a third party user 140 via data transmission to the affiliatedcolor service 105), from another system, or generated randomly. In someembodiments, the input color can be extracted from an image, such as animage provided by a user or an image of an item in an item catalog or ona network site. For more details on extracting colors from an image, seeU.S. patent application Ser. No. 14/316,483, entitled “IMAGE-BASED COLORPALETTE GENERATION,” filed on Jun. 26, 2014, and corresponding toAttorney Docket No. SEAZN.912A1, which is incorporated by referenceherein in its entirety.

In some embodiments, the input color is determined based at least inpart on preferences, behavior, or properties of a user. A system cananalyze properties of the user and determine one or more colors that theuser is likely to prefer. For example, the affiliated color service 105,or other system, may have access to a user profile that includes, forexample and without limitation, media preferences (e.g., preferredmovies, TV shows, books, music, etc.), purchase history (e.g., itemspurchased in an electronic marketplace), browse history (e.g., itemsbrowsed in an electronic marketplace), demographics (e.g., age, gender,nationality, etc.), geographical location (e.g., where the user residesand/or previously resided), item preferences (e.g., through the use ofwish lists), and the like. The affiliated color service 105 can analyzesuch information and determine probable colors that the user would like.One or more of these colors determined by the affiliated color service105 can be used as the input color. As another example, the affiliatedcolor service 105, or other system, can compare users to determine aninput color. For a particular user, the affiliated color service 105 cananalyze the color preferences of similar users (e.g., where similarityof users can be based at least in part on purchase history, mediapreferences, demographics, etc.) to determine one or more input colorsthat the particular user would prefer. This can be done by identifyinganother user with similar preferences and/or by aggregating user profileinformation to identify color preferences for an average user withsimilar preferences.

In block 802, analyzing the input color image can include determiningthe components of the color, such as the primary color values (e.g., RGBvalues), the luminance-chrominance values (e.g., YUV or YCbCr values),or the like. Analyzing the input color image can also includedetermining a threshold within which a color will be considered to bethe same as, or sufficiently similar to, the input color. The thresholdcan be based on color distance according to a color distance formula(e).An example of such a formula is based on a human perceptible colordifference. For more information on how to determine a human perceptiblecolor difference and the human color distance formula, see U.S. patentapplication Ser. No. 14/316,483, entitled “IMAGE-BASED COLOR PALETTEGENERATION,” filed on Jun. 26, 2014, and corresponding to AttorneyDocket No. SEAZN.912A1, which is incorporated by reference herein in itsentirety.

At block 804, the affiliated color service 105 identifies a plurality ofcolor palettes that include the input color. The color palettes can beprovided by the color palette providers 150. In some embodiments, one ormore color palettes can be provided by the palette data store 110 wherethe palettes stored therein have been voted on, ranked, and/or rated. Insome embodiments, the plurality of color palettes can be provided by thethird party users 140.

The affiliated color service 105 determines that a palette contains theinput color when that palette has at least one color that falls withinthe threshold color distance from the input color, as determined atblock 802. In this way, palettes that include colors that are notidentical to the input color but that include colors that aresufficiently close to the input color are included in the list ofpalettes identified at block 804. In some embodiments, the affiliatedcolor service 105 can use a fast color indexing routine to pull colorsfrom the palettes, as described in U.S. patent application Ser. No.14/315,700, entitled “FAST COLOR SEARCHING,” filed on Jun. 26, 2014, andcorresponding to Attorney Docket No. SEAZN.911A, which is incorporatedby reference herein in its entirety.

At block 806, the affiliated color service 105 generates a list ofaffiliated colors. The list of affiliated colors comprises the colorsfrom the list of palettes identified at block 804, excluding the inputcolor and those colors that are sufficiently close to the input color.In some embodiments, the list of affiliated colors can include all thecolors from the list of palettes. In certain embodiments, the list ofaffiliated colors is concatenated based at least in part on a thresholdnumber of colors to include in the list, a desired or targeted colorvariety, color exclusion rules, or the like.

At block 808, the affiliated color service 105 assigns weights to eachcolor in the list of affiliated colors. The affiliated color service 105loops through each color in the list, identifies from which palette thecolor originated, and adjusts a weight of the color based at least inpart on the ranking, rating, and/or number of votes associated with theoriginating palette. Adjusting the weight of the color can includeincreasing the weight factor by a number of votes or average rating ofthe originating palette. In some embodiments, adjusting the weight ofthe color includes scaling the ranking, rating, and/or number of votesbased at least in part on a number of factors that can include whichusers voted on the palette, the age of the palette, the number ofcomments on the palette, the geographical location of the voters, andthe like. In some embodiments, the ranking, rating, and/or voting of apalette is distributed among the colors within a particular palettebased at least in part on color popularity, ranking of colors within apalette, or the like. In certain embodiments, each color in a palettehas an individual ranking, rating, and/or number of votes where therating can be associated with the rating of the palette or independentfrom the rating of the palette.

Where a color appears in more than one palette, the weight of that colorcan be determined at least in part by aggregating the weights from eachoriginating palette. One aggregation method is to add the weights ofeach originating palette. As an example of a simple case, where a colorappears in 3 palettes, the weight of that color can be equal to the sumof the votes of each of the 3 palettes. It is to be understood thatother weight aggregation schemes can be used without departing from thescope of this disclosure. For example, weights can be aggregated using aweighted average of votes, an arithmetic mean of votes, or using someother algorithm (where votes can be the number of votes for a palette,the average rating of a palette, or the ranking of a palette). Weightaggregation can also be configured to account for rating of a palette orcolor as a function of time and/or geographical location.

The affiliated color service 105 can use a similar color distancethreshold when aggregating weights for an affiliated color. For example,a color distance threshold can be used such that when the affiliatedcolor service 105 is determining a weight for an affiliated color, itaggregates the votes from the palettes containing that affiliated colorand the palettes that contain a color with a distance from theaffiliated color that is less than or equal to the color distancethreshold. The color distance threshold used in analyzing the inputcolor at block 802 and the color distance threshold used in weightingaffiliated colors at block 808 can be the same or different and may bebased on the same or different color distance algorithms.

At block 810, the affiliated color service 105 tallies the weights ofeach color and provides a weighted, ordered, and/or ranked list ofaffiliated colors, where the rank of an affiliated color is based atleast in part on the relative weight of the color. The affiliated colorservice 105 can concatenate the ordered list of affiliated colors basedat least in part on a desired or targeted number of colors to include inthe list, a threshold weight factor to include in the list, a variety ofcolors in the list, color exclusion rules, or the like.

The affiliated color service 105 ends the routine depicted in FIG. 8 atblock 812.

FIG. 9 illustrates an example list of affiliated colors 900 generated bythe example routine of FIG. 8. Starting with the input color 902, theaffiliated color service 105 identifies a number of palettes 904 a-904d, each of which includes the input color or a color sufficientlysimilar to the input color (e.g., where the color distance between thecolor in the palette and the input color is less than a color distancethreshold). The palettes 904 a-904 d can be from a data store of human-or machine-created color palettes, but that have been voted on, ranked,or rated by a community of users. As used herein, the terms votes,rating, and/or ranking are used to indicate that there is a valueassociated with the palette where the value is indicative of a level ofhuman preference for the palette. Where only a single term is used(e.g., only vote, ranking, or rating), it is to be understood that theother terms could also be used. The rating of a color palette can bebased on a number of votes, such as where a palette's score can beincremented by a value according to a positive vote by a user, orsimilarly decremented by a value according to a negative vote by a user.Similarly, the rating of a color palette can be based on a rating systemwhere users can rate palettes on a rating scale (e.g., 0 to 5, 1 to 5, 0to 10, −5 to 5, etc.). Likewise, the rating of a color palette can bebased on users ranking palettes relative to one another.

Each time a palette is identified that contains the input color, or acolor sufficiently close to the input color, each of the other colors inthe palette is added to a list of affiliated colors. Each of the colorson the list of affiliated colors receives a weight that corresponds tothe rating of the originating palette. For example, each of colors 1-4in palette 1 904 a is assigned a weight corresponding to the number ofvotes for palette 1 904 a, represented by the variable A. Likewise, eachof the colors in palettes 2-4 904 b-904 d is assigned a weightcorresponding to the number of votes for each palette, represented bythe variables B-D.

If a color is found in more than one palette, the weight of the color isadjusted based on the rating of each of the originating palettes. Forexample, color 1 is found in palette 1 and palette 3, so the weight ofcolor 1 is based on the number of votes A+C. In some embodiments, anaffiliated color is considered to be found on another palette (e.g.,other than its originating palette) when the color in the other palettehas a color distance that is less than a color distance threshold fromthe affiliated color. In some embodiments, the weighting of colors isanalyzed as a function of position in a color space where thedistribution of weighted colors is analyzed to determine maximums. Thismay be able to provide additional details about which colors aregenerally considered to go well with the input color, which may resultin the ability to provide multiple suggestions of colors within and/oraround a peak color in the color space.

In some embodiments, the list of affiliated colors is adjusted tocombine colors that are close to one another based at least in part on acolor distance formula. This element can be used to decrease the numberof different colors where some colors are sufficiently similar to oneanother. This can be used to reduce the number of variations of a color,especially where the variations are insignificant, imperceptible, orotherwise lacking in value to a user.

Once the weights of each affiliated color is determined, a weighted orordered list of affiliated colors 906 can be provided. The ordered listcan include all the different colors from the palettes 904 a-904 d or itcan contain a subset of these colors. The ordering of the list can bebased at least in part on the weight of each color (e.g., higherweighted colors are ranked higher). As illustrated in the figure, thecolors 1-11 are ranked according to the sum of the votes for each color,where A is greater than B, which is greater than C, which is greaterthan D. Accordingly, using an aggregation scheme based on the sum ofweights, color 6 has a weight of B+C+D, color 1 has a weight of A+C,color 2 has a weight of A+D, etc.

In some embodiments, the ratings of the palettes are time-dependent. Theaffiliated color service 105 can use the time-dependent ratings toidentify trends in color combinations and/or to identify colorcombinations that are relatively stable over time (e.g., classic colorcombinations). This can also be used to determine color combinationsthat were popular at a particular time.

Time-dependent ratings can also be used to predict color trends in thefuture. For example, the most popular colors can be tracked as afunction of time, with the result plotted in a color space. The paththrough the color space of the most popular color (e.g., which color isthe most popular color at a point in time) can be used to project whatthe most popular color will be at some point in the future. For example,the affiliated color service 105 can determine a directional shift ofthe most popular color within the color space as a function of time andproject based on the directional shift. The affiliated color service 105can then provide suggestions of color combinations based on projectionsof which colors will be popular at a point in time. This may be useful,for example, for visual artists to predict trends in colors and/or colorcombinations. It is to be understood that this projection technique canbe used for the most popular color as well as the second most popular,third most popular, etc. It is also to be understood that thisprojection technique can be used for color palettes as well asindividual colors.

In some embodiments, geographical information can be included with theratings of the palettes (e.g., ratings of a palette can be provided as afunction of location of the voting user). The affiliated color service105 can use the geography-associated ratings to identify colorcombinations that are generally associated with a geographical location.For example, users within a geographical region may prefer a colorcombination associated with a sports team from that geographical region.The affiliated color service 105 can use information about the user(e.g., where the user is located) to suggest color combinations thatutilize the geography-associated ratings of palettes.

Example Affiliated Color Palette Generation Process

FIG. 10 illustrates a flow diagram of an affiliated color palettegeneration routine implemented by an affiliated color service 105. Theaffiliated color service 105 begins the routine at block 1000. At block1002, the affiliated color service 105 analyzes an input color orcolors. The input color or colors can be received as described abovewith respect to block 802 of the affiliated color routine illustrated inFIG. 8. Analysis of the input color or colors can include the functionsdescribed herein with reference to block 802 in FIG. 8. Returning toFIG. 10, in some embodiments, the routine can accept a plurality ofinput colors in determining a color palette, a particular example ofwhich is described in U.S. patent application Ser. No. 14/316,292,entitled “BUILDING A PALETTE OF COLORS BASED ON HUMAN COLORPREFERENCES,” filed on Jun. 26, 2014, and corresponding to AttorneyDocket No. SEAZN.904A1, which is incorporated by reference herein in itsentirety. In such a case, analyzing the input colors can includerepeating, for each input color, the element of analyzing the inputcolor as described with reference to block 802 in FIG. 8.

With reference to FIG. 10, at block 1004, the affiliated color service105 performs the routine depicted and described herein with reference toFIG. 8. In some embodiments, the affiliated color service 105 can moveto block 804 upon entering the routine because the input color or colorshas been analyzed in block 802. As described, the output of the routinedepicted in FIG. 8 is a weighted or ordered list of affiliated colors.The routine depicted in FIG. 8 can be modified to provide the orderedlist of affiliated colors where there is a plurality of input colors.For example, the affiliated color service 105 can identify palettes thatinclude one or more of the plurality of affiliated colors. In someembodiments, the affiliated color service 105 identifies palettes thatinclude all of the input colors. In some embodiments, the affiliatedcolor service 105 identifies palettes that include at least one of theinput colors. Once the palettes are identified, the weighting of thelist of affiliated colors can proceed much the same way as describedherein. In some embodiments, the weighting scheme of the affiliatedcolors is modified based on the input colors. For example, thepopularity of each of the input colors can be used to normalize or scalethe weighting factors of palettes of the respective input colors.

At block 1006, the affiliated color service 105 selects a color from theordered list of affiliated colors. The selection can be based on userinput through a user interface. For example, the routine can beinteractive such that a user provides the affiliated color service 105with the input color(s) and is provided the ordered list of affiliatedcolors generated at block 1004. The user can then select a color fromthe ordered list and indicate the selection to the affiliated colorservice 105. In some embodiments, the affiliated color service 105selects a color from the ordered list of affiliated colors. Theselection can be based at least in part on any one or more of theweighting of the affiliated color, the color distance of the affiliatedcolor from the input color and/or other affiliated colors on the list, aselection criteria determined by a user or other system, or the like.Selection of the color from the ordered list of affiliated colors addsthat color to the palette that includes the input color(s). In someembodiments, a plurality of colors can be selected.

In some embodiments, the affiliated color service 105 can select two ormore colors from the ordered list of affiliated colors and provide aprojection of what palettes would look like based on the selectedcolors. For example, the affiliated color service 105 can select the twomost popular colors from the ordered list of affiliated colors (e.g.,the two colors with the highest weight) and create a first tentativecolor palette that contains the input color(s) and the first popularcolor and a second tentative color palette with the input color(s) andthe second popular color. The affiliated color service 105 can thenprovide a weighted or ordered list of affiliated colors for each of thetwo tentative color palettes. This can be accomplished, for example,through the routine depicted and described herein with reference to FIG.8. This can advantageously be used to show a direction a color palettemay go based on selection of a particular color for a color palette. Incertain embodiments, the affiliated color service 105 can select the twoor more colors based on user input, input from another system, randomselection, or any combination of these.

In various embodiments, the affiliated color service 105 can recursivelyprovide tentative color palettes to provide a variety of tentative colorpalettes. For example, for the two tentative color palettes describedabove, the affiliated color service 105 can select the two most popularcolors from the ordered list of affiliated colors associated with eachtentative color palette to generate a second level of tentative colorpalettes. This process can continue a number of times to provide thevariety of tentative palettes. The affiliated color service 105 canselect a different number of colors in each iteration of this routineand for each tentative palette. The affiliated color service 105 canselect the two or more colors based on criteria that includes, forexample and without limitation, color popularity, color variety,exclusion rules, color distance, or any combination of these.

At block 1008, the affiliated color service 105 determines whetheranother color will be added to the palette. If another color is to beadded, routine returns to block 1002 to analyze the input colors, whichnow include the color selected in block 1006.

If the palette is finished (e.g., no other colors are to be added to thepalette), the affiliated color service 105 proceeds to block 1010 wherethe affiliated color service 105 provides the palette of affiliatedcolors. As illustrated in FIG. 1, the provided palette of affiliatedcolors can be stored in palette data store 110, sent to the third partyusers 140, and/or sent to the color palette providers 150. Theaffiliated color service 105 ends the routine at block 1012.

FIG. 11 illustrates example affiliated color palettes 1100 a and 1100 bgenerated by the routine of FIG. 10. Starting with an input color 902, afirst ordered list of affiliated colors 906 a is generated, as describedwith reference to FIG. 9. A color can be selected from the first orderedlist of affiliated colors 906 a. This results in a color palette 1100 athat includes the input color 902 and the selected color 908 (e.g.,color 2 from the first ordered list of affiliated colors). The new colorpalette 1100 a can then be used to generate a second ordered list ofaffiliated colors that now includes colors associated with the inputcolor 902 and the selected color 908. Because new and/or differentpalettes have been included in the routine to determine the ordered listof affiliated colors, the order of the affiliated colors may change. Inaddition, the second affiliated color list 906 b may include colors notpresent in the first ordered list of affiliated colors 906 a. Thisprocess can be repeated to build up a color palette with a plurality ofcolors, such as color palette 1100 b.

In some embodiments, as the number of input colors increases, the numberof affiliated colors decreases. The affiliated color service 105, forexample, may limit the identified palettes to those that include all ofthe input colors or a majority of the input colors. This may desirableto reduce the number of potential colors in the affiliated color listthat may result in visually unappealing color combinations. In this way,the palette generation is self-limiting as the more colors in thepalette the fewer affiliated colors that are presented.

By using the votes of a community of users, the generated color palettesrepresent a subjective color combination that may be different from whata mathematical formula may provide and that is generally preferable tousers. Using human arbiters to generate color combinations can providecolor palettes that mathematical algorithms may be incapable of fullydetermining.

The input color or colors can be determined or provided by a user, acomputer system, or a combination of both. For example, a user may wantto build an outfit that includes the color pink. Based on this inputcolor, the affiliated color service 105 can provide an ordered list ofaffiliated colors that have been determined by a community of users togo well with the input color. As another example, a user can select thecolor pink as an input color and the affiliated color service 105 can beconfigured to select a one or more additional colors randomly or throughanother algorithm to provide a suggested color palette (or multiplecolor palettes). As another example, the user can pick an item (or acollection of items) and a color fingerprint (or a subset of it) can beused as the basis for determining affiliated colors. For an example ofextracting a color fingerprint from an image of an item, see U.S. patentapplication Ser. No. 14/316,483, entitled “IMAGE-BASED COLOR PALETTEGENERATION,” filed on Jun. 26, 2014, and corresponding to AttorneyDocket No. SEAZN.912A1, which is incorporated by reference herein in itsentirety. Any of these color palettes could then be used to identifyand/or filter potential items for the user.

For more example details on generating affiliated color palettes, seeU.S. patent application Ser. No. 14/316,292, entitled “BUILDING APALETTE OF COLORS BASED ON HUMAN COLOR PREFERENCES,” filed on Jun. 26,2014, and corresponding to Attorney Docket No. SEAZN.904A1, which isincorporated by reference herein in its entirety.

Depending on the embodiment, certain acts, events, or functions of anyof the algorithms described herein can be performed in a differentsequence, can be added, merged, or left out altogether (e.g., not alldescribed acts or events are necessary for the practice of thealgorithm). Moreover, in certain embodiments, acts or events can beperformed concurrently, e.g., through multi-threaded processing,interrupt processing, or multiple processors or processor cores or onother parallel architectures, rather than sequentially.

The various illustrative logical blocks, modules, and algorithm elementsdescribed in connection with the embodiments disclosed herein can beimplemented as electronic hardware, computer software, or combinationsof both. To clearly illustrate this interchangeability of hardware andsoftware, various illustrative components, blocks, modules, and elementshave been described above generally in terms of their functionality.Whether such functionality is implemented as hardware or softwaredepends upon the particular application and design constraints imposedon the overall system. The described functionality can be implemented invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the disclosure.

The various illustrative logical blocks and modules described inconnection with the embodiments disclosed herein can be implemented orperformed by a machine, such as a general purpose processor, a digitalsignal processor (DSP), an application specific integrated circuit(ASIC), a field programmable gate array (FPGA) or other programmablelogic device, discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. A general purpose processor can be a microprocessor,but in the alternative, the processor can be a controller,microcontroller, or state machine, combinations of the same, or thelike. A processor can also be implemented as a combination of computingdevices, e.g., a combination of a DSP and a microprocessor, a pluralityof microprocessors, one or more microprocessors in conjunction with aDSP core, or any other such configuration.

The elements of a method, process, or algorithm described in connectionwith the embodiments disclosed herein can be embodied directly inhardware, in a software module executed by a processor, or in acombination of the two. A software module can reside in RAM memory,flash memory, ROM memory, EPROM memory, EEPROM memory, registers, harddisk, a removable disk, a CD-ROM, or any other form of computer-readablestorage medium known in the art. An exemplary storage medium can becoupled to the processor such that the processor can read informationfrom, and write information to, the storage medium. In the alternative,the storage medium can be integral to the processor. The processor andthe storage medium can reside in an ASIC. The ASIC can reside in a userterminal. In the alternative, the processor and the storage medium canreside as discrete components in a user terminal. A software module cancomprise computer-executable instructions that cause a hardwareprocessor to execute the computer-executable instructions. Thecomputer-executable instructions can comprise a scripted computerlanguage and/or a compiled computer language. Computer-executableinstructions can comprise, for example and without limitation,JAVASCRIPT®, PYTHON™, php, SQL, C, C++, JAVA®, C#, Fortran, BASIC, shellscripts, Perl, or the like.

Conditional language used herein, such as, among others, “can,” “might,”“may,” “e.g.,” and the like, unless specifically stated otherwise, orotherwise understood within the context as used, is generally intendedto convey that certain embodiments include, while other embodiments donot include, certain features, elements and/or states. Thus, suchconditional language is not generally intended to imply that features,elements and/or states are in any way required for one or moreembodiments or that one or more embodiments necessarily include logicfor deciding, with or without author input or prompting, whether thesefeatures, elements and/or states are included or are to be performed inany particular embodiment. The terms “comprising,” “including,”“having,” “involving,” and the like are synonymous and are usedinclusively, in an open-ended fashion, and do not exclude additionalelements, features, acts, operations, and so forth. Also, the term “or”is used in its inclusive sense (and not in its exclusive sense) so thatwhen used, for example, to connect a list of elements, the term “or”means one, some, or all of the elements in the list.

Disjunctive language such as the phrase “at least one of X, Y or Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y or Z, or any combination thereof (e.g., X, Y and/or Z).Thus, such disjunctive language is not generally intended to, and shouldnot, imply that certain embodiments require at least one of X, at leastone of Y or at least one of Z to each be present.

Unless otherwise explicitly stated, articles such as “a” or “an” shouldgenerally be interpreted to include one or more described items.Accordingly, phrases such as “a device configured to” are intended toinclude one or more recited devices. Such one or more recited devicescan also be collectively configured to carry out the stated recitations.For example, “a processor configured to carry out recitations A, B andC” can include a first processor configured to carry out recitation Aworking in conjunction with a second processor configured to carry outrecitations B and C.

While the above detailed description has shown, described, and pointedout novel features as applied to various embodiments, it will beunderstood that various omissions, substitutions, and changes in theform and details of the devices or algorithms illustrated can be madewithout departing from the spirit of the disclosure. As will berecognized, certain embodiments described herein can be embodied withina form that does not provide all of the features and benefits set forthherein, as some features can be used or practiced separately fromothers. All changes that come within the meaning and range ofequivalency of the claims are to be embraced within their scope.

1. A computer-implemented method for generating a custom color palette,the computer-implemented method comprising: under control of a hardwarecomputing device configured with specific computer-executableinstructions, deriving a first color from one or more color images, thefirst color associated with a color name; obtaining a plurality ofaffiliated palettes, wherein each affiliated palette of the plurality ofaffiliated palettes includes the first color; identifying at least onesecond color based at least in part on the plurality of affiliatedpalettes, wherein at least a subset of the plurality of affiliatedpalettes includes a color corresponding to the at least one secondcolor; and generating the custom palette to include at least the firstcolor and the second color.
 2. The computer-implemented method of claim1, wherein each affiliated palette of the plurality of affiliatedpalettes is associated with a respective weight, and wherein identifyingthe at least one second color further comprises: identifying the subsetof the plurality of affiliated palettes; determining a cumulative weightfor the at least one second color based at least on respective weightsassociated with each palette of the subset; and determining that thecumulative weight has a higher rank than another weight.
 3. Thecomputer-implemented method of claim 1, wherein obtaining the pluralityof affiliated palettes further comprises: determining that eachaffiliated palette, of the plurality of affiliated palettes, includesthe first color by including a color that is within a threshold distancefrom the first color in a color space.
 4. The computer-implementedmethod of claim 1, further comprising: receiving a reference color namedifferent from the color name; and determining, from the reference colorname, the color name based at least in part on natural languageprocessing.
 5. The computer-implemented method of claim 1, furthercomprising: receiving the color name; and identifying the one or morecolor images associated with the color name.
 6. A non-transitorycomputer-readable storage medium storing computer-executableinstructions that when executed by a processor perform operationscomprising: deriving a first color from one or more color images;obtaining a plurality of affiliated palettes, wherein each affiliatedpalette of the plurality of affiliated palettes includes the firstcolor; identifying at least one second color based at least in part onthe plurality of affiliated palettes, wherein at least a subset of theplurality of affiliated palettes includes a color corresponding to theat least one second color; and generating a custom palette to include atleast the first color and the second color.
 7. The non-transitorycomputer-readable storage medium of claim 6, wherein each affiliatedpalette of the plurality of affiliated palettes is associated with arespective weight, and wherein identifying the at least one second colorfurther comprises: identifying the subset of the plurality of affiliatedpalettes; determining a cumulative weight for the at least one secondcolor based at least on respective weights associated with each paletteof the subset; and determining that the cumulative weight has a higherrank than another weight.
 8. The non-transitory computer-readablestorage medium of claim 6, wherein obtaining the plurality of affiliatedpalettes further comprises: determining that each affiliated palette, ofthe plurality of affiliated palettes, includes the first color byincluding a color that is within a threshold distance from the firstcolor in a color space.
 9. The non-transitory computer-readable storagemedium of claim 6, wherein the operations further comprise: receiving areference color name different from the color name; and determining,from the reference color name, the color name based at least in part onnatural language processing.
 10. The non-transitory computer-readablestorage medium of claim 6, wherein the operations further comprise:receiving the color name; and identifying the one or more color imagesassociated with the color name.
 11. The non-transitory computer-readablestorage medium of claim 10, wherein each image of the one or more colorimages is associated with metadata, and wherein identifying the one ormore color images further comprises: determining, from a plurality ofcolor images, that respective metadata of the one or more color imagescorresponds to the color name.
 12. The non-transitory computer-readablestorage medium of claim 6, wherein the operations further comprise:identifying first item data associated with the first color; identifyingsecond item data associated with the second color; and providing thefirst item data and the second item data to a computing device.
 13. Asystem comprising: a data store configured to at least storecomputer-executable instructions; and a hardware processor incommunication with the data store, the hardware processor configured toexecute the computer-executable instructions to at least: receive afirst color; obtain a plurality of affiliated palettes, wherein eachaffiliated palette of the plurality of affiliated palettes includes thefirst color; identify at least one second color based at least in parton the plurality of affiliated palettes, wherein at least a subset ofthe plurality of affiliated palettes includes a color corresponding tothe at least one second color; and generate a custom palette to includeat least the first color and the second color.
 14. The system of claim13, wherein each affiliated palette of the plurality of affiliatedpalettes is associated with a respective weight, and wherein identifyingthe at least one second color further comprises: identifying the subsetof the plurality of affiliated palettes; determining a cumulative weightfor the at least one second color based at least on respective weightsassociated with each palette of the subset; and determining that thecumulative weight has a higher rank than another weight.
 15. The systemof claim 13, wherein obtaining the plurality of affiliated palettesfurther comprises: determining that each affiliated palette, of theplurality of affiliated palettes, includes the first color by includinga color that is within a threshold distance from the first color in acolor space.
 16. The system of claim 13, wherein the hardware processoris further configured to execute the computer-executable instructions toat least: receive a color name; and determine the first color based atleast in part on the color name.
 17. The system of claim 16, wherein thehardware processor is further configured to execute thecomputer-executable instructions to at least: determine, from the colorname, a secondary color name based at least in part on natural languageprocessing, wherein the first color corresponds to the secondary colorname.
 18. The system of claim 13, wherein the hardware processor isfurther configured to execute the computer-executable instructions to atleast: identify a first image that includes the first color; andidentify a second image that includes the second color.
 19. The systemof claim 18, wherein the hardware processor is further configured toexecute the computer-executable instructions to at least: identify firstitem data associated with the first image; identify second item dataassociated with the second image; and provide the first item data andthe second item data to a computing device.
 20. The system of claim 18,wherein identifying that the first image includes the first colorfurther comprises: determining that the first image includes the firstcolor by including a color that is within a threshold distance from thefirst color in a color space.